All Apps and Add-ons

Field Extraction Regex - optional fields

atornes
Path Finder

I'm terrible with Regex's and can't figure out what is wrong with mine...

I have this Regex:

(?P<date>[^,\n]+),\d*--(?P<account>[^,]+),(?P<provider>[^,]+),(?P<product>[^,]+),(?P<service_username>[^,]*),(?P<billing_status>[^,]+),(?P<activities>[^,]*),(?P<requests>[^,]*),(?P<days>[^,]*),(?P<ruuid>[^,]*)

Currently working fine for events like:

2014-04-03,1034--account,provider,product,username,Billable,10073107,,19,e5xwcmhbrk,

We've added another field to the end that is optional for some events (80--parent below):

2014-04-03,1307--account,provider,product,username,Billable,1614,0,,,80--parent

I can't figure out how to extract that "parent" value when it exists (I don't want the preceding "80--". It should work exactly as the 1304--account works in the currently functioning regex. When I try to essentially copy that part of the functioning regex, my queries only return the events that have that parent field name

0 Karma
1 Solution

somesoni2
Revered Legend

Try this.

"^(?P<date>[^,\n]+),\d+--(?P<account>[^,]+),(?P<provider>[^,]+),(?P<product>[^,]+),(?P<service_username>[^,]*),(?P<billing_status>[^,]+),(?P<activities>[^,]*),(?P<requests>[^,]*),(?P<days>[^,]*),(?P<ruuid>[^,]*)[,\d-]*(?P<parent>.*)"

Updated answer

This will be scalable. For every new field to be added at the end of event, your can add "[,\d-]*(?P<newFieldName>[^,]*)" to the regex.

"^(?P<date>[^,\n]+),\d+--(?P<account>[^,]+),(?P<provider>[^,]+),(?P<product>[^,]+),(?P<service_username>[^,]*),(?P<billing_status>[^,]+),(?P<activities>[^,]*),(?P<requests>[^,]*),(?P<days>[^,]*),(?P<ruuid>[^,]*)[,\d-]*(?P<parent>[^,]*)"

View solution in original post

somesoni2
Revered Legend

Try this.

"^(?P<date>[^,\n]+),\d+--(?P<account>[^,]+),(?P<provider>[^,]+),(?P<product>[^,]+),(?P<service_username>[^,]*),(?P<billing_status>[^,]+),(?P<activities>[^,]*),(?P<requests>[^,]*),(?P<days>[^,]*),(?P<ruuid>[^,]*)[,\d-]*(?P<parent>.*)"

Updated answer

This will be scalable. For every new field to be added at the end of event, your can add "[,\d-]*(?P<newFieldName>[^,]*)" to the regex.

"^(?P<date>[^,\n]+),\d+--(?P<account>[^,]+),(?P<provider>[^,]+),(?P<product>[^,]+),(?P<service_username>[^,]*),(?P<billing_status>[^,]+),(?P<activities>[^,]*),(?P<requests>[^,]*),(?P<days>[^,]*),(?P<ruuid>[^,]*)[,\d-]*(?P<parent>[^,]*)"

carasso
Splunk Employee
Splunk Employee

The correct regex is:

"(?P[^,n]+),\d*--(?P[^,]+),(?P[^,]+),(?P[^,]+),(?P[^,]),(?P[^,]+),(?P[^,]\
*),(?P[^,]
),(?P[^,]),(?P[^,])(?:,\d*--(?P.*))?"

The entire extra argument is encased in one "(?: ... )?". The other regexes suggested above allow invalid things, such as just "5-5,5-5,parent"

0 Karma

somesoni2
Revered Legend

For the events where this field is not present,comma is also not present so its options here. See the updated answer for scalable regex.

0 Karma

atornes
Path Finder

Works. Will this easily allow me to add other fields? Seems weird that its not comma-delimited like everything else

0 Karma
Get Updates on the Splunk Community!

Shape the Future of Splunk: Join the Product Research Lab!

Join the Splunk Product Research Lab and connect with us in the Slack channel #product-research-lab to get ...

Auto-Injector for Everything Else: Making OpenTelemetry Truly Universal

You might have seen Splunk’s recent announcement about donating the OpenTelemetry Injector to the ...

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...