Splunk Search

How to configure props and transforms.conf based on the rex extractions in my sample search?

Path Finder

I need to construct props and transforms for below sample search.

index=blaa  sourcetype=my_source | rex field=X__Edgescape "lat=(?P(.*?)),"  
| rex field=X__Edgescape "long=(?P(.*?)),"  
| rex field=X__Edgescape "continent=(?P(.*?))," 
| rex field=X__Edgescape "country_code=(?P(.*?))," 
| table 

i constructed below one too work but just thought of take expert advice.

transforms.conf

[latitude1]
SOURCE_KEY = X_Edgescape 
REGEX ="lat=(?P(.*?)),"  
MV_ADD = TRUE 
[longitude1]
SOURCE_KEY = X_Edgescape 
REGEX ="lat=(?P(.*?)),"  
MV_ADD = TRUE 

PROPS.CONF

[sourcetype]
REPORT-fields = latitude1,longitude1
0 Karma

Esteemed Legend

The problem is that you are including double-quotes in your definition. This is wrong:

 REGEX = "country_code=(?P(.*?)),"

And should be this:

 REGEX = country_code=([^,]*),

Yes, it is really that simple.

0 Karma

SplunkTrust
SplunkTrust

Good spot! You still have to either remove the ?P because that's the start of a named capture group, and add FORMAT = yourfieldname::$1, or just add the fieldname after the ?P, like (?P<yourfieldname>yourregex)

0 Karma

Esteemed Legend

Good catch. I am pretty sure that having the ?P in there will break it. I have modified my answer.

0 Karma

Esteemed Legend

You are missing a FORMAT line, something like this:

FORMAT = field1::$1
0 Karma

Path Finder

I have tried above one but it did not worked ..just thought of posting again ..when i run the search its working fine no issues .

sourcetype=akamai:syslog
|rex field=X_Akamai_Edgescape "country_code=(?P(.*?)),"  
|rex field=X_Akamai_Edgescape "lat=(?P(.*?)),"
|rex field=X_Akamai_Edgescape "continent=(?P(.*?)),"

here is my code ..

PROPS.CONF 
[akamai:syslog]
REPORT-fields = country_code

transforms.conf
[country_code]
SOURCE_KEY = X_Akamai_Edgescape 
REGEX ="country_code=(?P(.*?)),"   
MV_ADD = TRUE
0 Karma

SplunkTrust
SplunkTrust

As written in my answer above - your regexes are still missing a capture group name, so Splunk doesn't know which field these matches should be extracted too 😉

0 Karma

SplunkTrust
SplunkTrust

Is anyone else picking up on the [longitude1] config in the transforms.conf section above? It has lat=, which will probably not get a very good longitude. Perhaps that is just a typo in the Question, but not in the actual .conf file.

SplunkTrust
SplunkTrust

Probably a typo, but field name in your search query has two underscore (transforms.conf has one) and the name capturing group is missing. but if it works (and your conf files has correct names), it looks good.

Path Finder

Thanks,
but i was expecting some advanced kind of answer.
Im aware of my field names all.

0 Karma

SplunkTrust
SplunkTrust

If you want some advanced kind of answer, you should ask some advanced kind of question 😉
So far, your config is fine, besides the mentioned problems/improvements, so I don't know what exactly you're expecting.

0 Karma

SplunkTrust
SplunkTrust

Your regexes are missing the field name that the data should be extracted too, so instead of

REGEX ="lat=(?P(.*?)),"  

do

REGEX ="lat=(?P<yourfieldnamehere>(.*?)),"  

That should work.

Hope that helps - if it does I'd be happy if you would upvote/accept this answer, so others could profit from it. 🙂