Splunk Search

Regex not extracting fields with spaces between fields

ssaenger
Communicator

Hi,

i am again struggling with regex.
I have the following lines in a log file, some of the text is constantly in the log file so i thought that i would be able to keep it, i think my issue is with the gaps between the text, i am using /s+ to cope with whitespaces and possible tabs. I would like to extract the alpha-numeric strings.

2015/07/14 14:06:27.572 [Logger] [qtp1455736317-371]:  INFO: [55a51763e4b0a933] PC Response: RequestId: 150316 User-Id: 04-7FFABFFF-00809F1D RequestType: Open

and i have tried to write the regex

(?<date_time>\d{4}\.\d{2}\.\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3})\s+[Logger]\s+(?<transaction_id>[^;]+)\s+INFO:\s+(?<info>[^;]+)\s+PC\s+Response:\s+RequestId:\s+(?<request_id>[^;]+)\s+ User-Id:\s+(?<user_id>[^;]+)\s+RequestType:\s+Open

none of the fields are picked up, i do not know if its because there are : in the text and they need to be escaped..

0 Karma
1 Solution

jeffland
SplunkTrust
SplunkTrust

This should work:

(?<date_time>\d{2}(?:\d{2}.)*\d{3})\s\[Logger\]\s\[(?<transaction_id>[^\]]+)\]\:\s+INFO\:\s+\[(?<info>[^\]]+)\]\s+PC\sResponse\:\s+RequestId\:\s+(?<request_id>[\S]+)\sUser\-Id\:\s+(?<user_id>[\S]+)\s+RequestType\:\s+Open

Some generally good ideas: Use tools to help you out, such as regex101.com - with their help, you don't need to poke around blindfolded and regular expressions become a fun thing. Furthermore, always escape special characters. Even if the tools may work without an escaped :, splunk is quite strict with that. Lastly, in your capturing groups you used

[^;]

to capure digits and other characters - I'd recommend you always use a negation of what comes before the next field, which is why I used \S for those fields that are separated by a whitespace and

[^\]]

for those fields which were in square brackets. If you want to have the square brackets in your fields, then simply change those parts so that they also use the above method.

View solution in original post

jeffland
SplunkTrust
SplunkTrust

This should work:

(?<date_time>\d{2}(?:\d{2}.)*\d{3})\s\[Logger\]\s\[(?<transaction_id>[^\]]+)\]\:\s+INFO\:\s+\[(?<info>[^\]]+)\]\s+PC\sResponse\:\s+RequestId\:\s+(?<request_id>[\S]+)\sUser\-Id\:\s+(?<user_id>[\S]+)\s+RequestType\:\s+Open

Some generally good ideas: Use tools to help you out, such as regex101.com - with their help, you don't need to poke around blindfolded and regular expressions become a fun thing. Furthermore, always escape special characters. Even if the tools may work without an escaped :, splunk is quite strict with that. Lastly, in your capturing groups you used

[^;]

to capure digits and other characters - I'd recommend you always use a negation of what comes before the next field, which is why I used \S for those fields that are separated by a whitespace and

[^\]]

for those fields which were in square brackets. If you want to have the square brackets in your fields, then simply change those parts so that they also use the above method.

ssaenger
Communicator

works a treat, thanks jeffland

0 Karma

sduff_splunk
Splunk Employee
Splunk Employee
(?<date_time>\d{4}/\d{2}/\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3})\s+[Logger]\s+(?<transaction_id>[^;]+)\s+INFO:\s+(?<info>[^;]+)\s+PC\s+Response:\s+RequestId:\s+(?<request_id>[^;]+)\s+User-Id:\s+(?<user_id>[^;]+)\s+RequestType:\s+Open

Your date_time has (slash). which matches only a dot, while your event has /. You also have a space before User-Id, while the \s+ means that there must be a second white space character there.

I think you got all the other s+ correct. I'm not sure why you are matching everything except sem-colons ([^;]) but it should still work.

0 Karma
Get Updates on the Splunk Community!

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Splunk App for Anomaly Detection End of Life Announcement

Q: What is happening to the Splunk App for Anomaly Detection?A: Splunk is officially announcing the ...

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...