topic Re: Field Extraction in Splunk Search

Field Extraction

shivam_j — Tue, 19 Nov 2019 15:44:46 GMT

Hi All, I want to extract the log to be extracted from error message till : message : , but not getting it, I have tried few queries but I am not getting the desired output:

error message System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote name could not be resolved: TimeoutChecklistExecutionResultHandler.cs:line 293 : message : {
Please suggest what query should I choose.
PS: | rex ".(Asterisk)error message (?.(Asterisk)) : message : .*" is not working

Re: Field Extraction

mayurr98 — Tue, 19 Nov 2019 17:52:13 GMT

try this:

| makeresults 
|  eval _raw="error message System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote name could not be resolved: TimeoutChecklistExecutionResultHandler.cs:line 293 : message : {" 
|  rex  "error\s+message\s+(?<Message>.+?(?=(\:\s+message\s+\:)))"

Re: Field Extraction

uagrawal_splunk — Tue, 19 Nov 2019 17:52:52 GMT

Try the below query
error message .*(?=(: message :))

Re: Field Extraction

uagrawal_splunk — Tue, 19 Nov 2019 18:10:35 GMT

I tried the query in regex101 and it takes 650 steps to match the regex. And the same query with little modification takes the 78 steps to match the regex. Here is the query:

error\s+message\s+(?<Message>.*(?=(\:\s+message\s+\:)))

Re: Field Extraction

mayurr98 — Tue, 19 Nov 2019 18:36:07 GMT

yes, there are several ways to solve the same problem but the idea behind it to use Positive Lookahead and Quantifier.

you could also try error\s+message\s+(?<Message>.+(?=(\:\s+message\s+\:)))

Re: Field Extraction

Anantha123 — Tue, 19 Nov 2019 18:36:57 GMT

Hi Try this

| rex field=_raw "error\smessage\s(?P<ErrorMessage>.*)\s:\smessage\s" | table ErrorMessage

Re: Field Extraction

dindu — Tue, 19 Nov 2019 20:12:10 GMT

Hi Shivam,

We could use regexes to extract the required values.
All the answers given above will give the desired results.However, it is advisable to achieve in minimum steps and minimum latency.
Otherwise it will have an impact on performance.

You could check your regex in https://regex101.com/ and look on the top middle to see the no of steps iterated and time consumed.
Its better not to include too much identifier for matching, as it will impact the performance.

I would suggest to go with the below regex which only took 46 steps- considering the key identifiers to be the words error message and message.
Your overall query becomes .

|Your_search
|rex field = _raw  "error\smessage(?P<exception_message>.*)message"
| table exception_message

Please accept and upvote the answer if this helped you.
Happy Splunking!!

Re: Field Extraction

shivam_j — Wed, 20 Nov 2019 07:51:38 GMT

Hi Dindu, unfortunately, the log already contains 'message' keyword in between (which i skipped in the log i mentioned in the que. otherwise it would get long). So, this won't work. Thanks for the answer bdw.