topic How to write regex to extract my field? in Splunk Search

How to write regex to extract my field?

tlow — Thu, 23 Oct 2014 20:39:26 GMT

Hi, i'm try using the interactive field extractor tools create a field for this

"Exception Message"="Thread was being aborted."

but getting following error "The generated regex was unable to match all examples (e.g., =, Thread was being aborted., Exception Message). Consider entering different examples, or manually editing the regex."

any suggestion a regex syntax or how to auto detect the field it?

Re: How to write regex to extract my field?

somesoni2 — Thu, 23 Oct 2014 20:48:57 GMT

Can you post some complete sample events, along with the what string to extract?

Re: How to write regex to extract my field?

tlow — Thu, 23 Oct 2014 20:54:37 GMT

"Exception Message"="Thread was being aborted.", Source=OAA.Security.PassiveSTS.Shell, TargetSite="Void SignOutRequest()", WindowsIdentity="IIS APPPOOL\Test.com"

Above is the logs, but splunk auto assign the field to "Source" ,"TargetSite" and "WindowsIdentity" but not "Exception Message" or maybe the "

i want extract exception message "Exception Message"="*" or better to have it in props.conf to auto assign it.
thanks

Re: How to write regex to extract my field?

somesoni2 — Thu, 23 Oct 2014 21:10:33 GMT

For Search time

your base search | rex field=_raw "\"Exception Message\"=\"(?<ExceptionMessage>[^\"]*)\""

In Props.conf

EXTRACT-exceptionmsg = \"Exception Message\"=\"(?<ExceptionMessage>[^\"]*)\"

Re: How to write regex to extract my field?

tlow — Thu, 23 Oct 2014 22:36:11 GMT

works great, but how to setup it up so that splunk automatic recognize fields based on

"Exception Message"="*"

for example splunk recognize this Source=OAA.Security.PassiveSTS.Shell automatic assigning "Source" as the field,
thank-you for your help

Re: How to write regex to extract my field?

martin_mueller — Thu, 23 Oct 2014 22:48:51 GMT

If you're asking about extracting the field name along with the field value, you can do that with a combination of props.conf and transforms.conf:

props.conf

[your_sourcetype]
REPORT-kv = extract_quoted_fields
...

transforms.conf

[extract_quoted_fields]
REGEX = "(?<_KEY_1>([^"]|\\")+)"\s*=\s*"(?<_VAL_1>([^"]|\\")+)"

Make sure my Q'n'D regex actually suits your data, didn't test it myself. The main thing is to use the magic _KEY_x and _VAL_x names to extract both field names and field values in one swoop.