topic Re: Regex not extracting fields with spaces between fields in Splunk Search

Regex not extracting fields with spaces between fields

ssaenger — Wed, 15 Jul 2015 08:35:00 GMT

Hi,

i am again struggling with regex.
I have the following lines in a log file, some of the text is constantly in the log file so i thought that i would be able to keep it, i think my issue is with the gaps between the text, i am using /s+ to cope with whitespaces and possible tabs. I would like to extract the alpha-numeric strings.

2015/07/14 14:06:27.572 [Logger] [qtp1455736317-371]:  INFO: [55a51763e4b0a933] PC Response: RequestId: 150316 User-Id: 04-7FFABFFF-00809F1D RequestType: Open

and i have tried to write the regex

(?<date_time>\d{4}\.\d{2}\.\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3})\s+[Logger]\s+(?<transaction_id>[^;]+)\s+INFO:\s+(?<info>[^;]+)\s+PC\s+Response:\s+RequestId:\s+(?<request_id>[^;]+)\s+ User-Id:\s+(?<user_id>[^;]+)\s+RequestType:\s+Open

none of the fields are picked up, i do not know if its because there are : in the text and they need to be escaped..

Re: Regex not extracting fields with spaces between fields

sduff_splunk — Wed, 15 Jul 2015 08:53:15 GMT

(?<date_time>\d{4}/\d{2}/\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3})\s+[Logger]\s+(?<transaction_id>[^;]+)\s+INFO:\s+(?<info>[^;]+)\s+PC\s+Response:\s+RequestId:\s+(?<request_id>[^;]+)\s+User-Id:\s+(?<user_id>[^;]+)\s+RequestType:\s+Open

Your date_time has (slash). which matches only a dot, while your event has /. You also have a space before User-Id, while the \s+ means that there must be a second white space character there.

I think you got all the other s+ correct. I'm not sure why you are matching everything except sem-colons ([^;]) but it should still work.

Re: Regex not extracting fields with spaces between fields

jeffland — Wed, 15 Jul 2015 08:59:01 GMT

This should work:

(?<date_time>\d{2}(?:\d{2}.)*\d{3})\s\[Logger\]\s\[(?<transaction_id>[^\]]+)\]\:\s+INFO\:\s+\[(?<info>[^\]]+)\]\s+PC\sResponse\:\s+RequestId\:\s+(?<request_id>[\S]+)\sUser\-Id\:\s+(?<user_id>[\S]+)\s+RequestType\:\s+Open

Some generally good ideas: Use tools to help you out, such as regex101.com - with their help, you don't need to poke around blindfolded and regular expressions become a fun thing. Furthermore, always escape special characters. Even if the tools may work without an escaped :, splunk is quite strict with that. Lastly, in your capturing groups you used

[^;]

to capure digits and other characters - I'd recommend you always use a negation of what comes before the next field, which is why I used \S for those fields that are separated by a whitespace and

[^\]]

for those fields which were in square brackets. If you want to have the square brackets in your fields, then simply change those parts so that they also use the above method.

Re: Regex not extracting fields with spaces between fields

ssaenger — Wed, 15 Jul 2015 11:07:15 GMT

works a treat, thanks jeffland