topic How to apply a regular expression that pulls multiple values from application log and show them to the given field name in Splunk Search

How to apply a regular expression that pulls multiple values from application log and show them to the given field name

iqbalintouch — Wed, 30 Sep 2020 05:37:11 GMT

Hi all,

I've been struggling to extract certain values from application logs and assign them to the given field name. As I don't know how to use or write regular expression in splunk, I need help to write a query to get the desired output. Here is my base search query:

https://www.myapplication.com/myapi/version5/autofill/ "ERROR"

here is the output log:

"ERROR" "store.view.app.api.controller.myClientLoggingController" "viewhost02" "myview2_2" <> "catalina-exec-7" "requestId=d4s6666-9d6e-2c0g-7c20-6e9f7wfa7f6" <> "clientIp=234.234.234.22" "store.view.app.api.controller.myClientLoggingController.logError(?:?):My-AngularApp
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxcxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

NOTE: in above log I have replaced the brackets <> with quotes ""

Now I want to extract the "requestId", "clientIp" and "My-AngularApp" and assign them to field name as "Req_ID", "Cust_IP" and "App_Name" respectively.

Can someone please help with the query to achieve the desired output, as I always struggle with REX syntax and can't write the query by my own.

Thank you in advance.

Re: How to apply a regular expression that pulls multiple values from application log and show them

richgalloway — Tue, 09 Jun 2020 16:59:44 GMT

I restored the angle brackets to make it easier for helpers to create a regex for you. Please edit the question to correct any mistakes I may have made.

Are you familiar with regex101.com? It's a great web site for testing regular expressions.

Re: How to apply a regular expression that pulls multiple values from application log and show them

Yorokobi — Tue, 09 Jun 2020 20:30:47 GMT

Here's a stab at it. You'll be better served by not substituting elements of your log except to obfuscate sensitive data. I worked with what you have, which means it may not work as-is but this should give you a pattern to follow.

<(?<log_level>[^>]+)>\s+<(?<class>[^>]+)>\s+<(?<hostname>[^>]+)>\s+<(?<viewname>[^>]+)>\s+<(?<unknown1>[^>].)?>\s+<(?<exec_process>[^>]+)>\s+<requestId=(?<Req_ID>[^>]+)>\s+<(?<unknown2>[^>]+)?>\s+<clientIp=(?<Cust_IP>[^\"]+)\">\"(?<log_class>[^:].+):(?<App_Name>[^\ ]+)

Regex101 link: https://regex101.com/r/FX8lkQ/1