Getting Data In

How to extract values for specific token from pipe delimited log

att35
Builder

Hi,

I have some application logs in the following format:

 

ERROR | 2021-07-20 06:55:54 EDT | Field1 = Value1 | Field2 = Value2 | Long Error String - Another long error string | Field3 = Value3 | ... | ... 

 

 

Most of the tokens are in Field=Value format and Splunk is able to extract them just fine except the portion where there is no Field listed. Just two different error strings separated by a " - ".  (These strings may contain other special characters as part of the error)

Is there a way I can extract both of them separately, e.g. signature_1, signature_2 without disturbing rest of the extractions? I would prefer doing this with props/transforms.

I was thinking of using "DELIMS" option but not sure how to target just that particular part of the log.

Labels (3)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @att35 

You can try inline rex as below, and props.conf shall be deployed to SearchHead.

<your_search>
| rex "^\w+\s+\|\s+\d+-\d+-\d+\s+\d+:\d+:\d+\s+\w+\s+\|\s+\w+\s+=\s+.+?\|\s+\w+\s+=\s+.+?\|(?<signature_1>.+?)\-(?<signature_2>.+?)\|"

 #props.conf

[your_sourcetype]
EXTRACT-sign = ^\w+\s+\|\s+\d+-\d+-\d+\s+\d+:\d+:\d+\s+\w+\s+\|\s+\w+\s+=\s+.+?\|\s+\w+\s+=\s+.+?\|(?<signature_1>.+?)\-(?<signature_2>.+?)\|

---

An upvote would be appreciated and Accept solution if this reply helps!

View solution in original post

venkatasri
SplunkTrust
SplunkTrust

Hi @att35 

You can try inline rex as below, and props.conf shall be deployed to SearchHead.

<your_search>
| rex "^\w+\s+\|\s+\d+-\d+-\d+\s+\d+:\d+:\d+\s+\w+\s+\|\s+\w+\s+=\s+.+?\|\s+\w+\s+=\s+.+?\|(?<signature_1>.+?)\-(?<signature_2>.+?)\|"

 #props.conf

[your_sourcetype]
EXTRACT-sign = ^\w+\s+\|\s+\d+-\d+-\d+\s+\d+:\d+:\d+\s+\w+\s+\|\s+\w+\s+=\s+.+?\|\s+\w+\s+=\s+.+?\|(?<signature_1>.+?)\-(?<signature_2>.+?)\|

---

An upvote would be appreciated and Accept solution if this reply helps!

att35
Builder

@venkatasri 

Thank you. Regex was able to extract both parts but I noticed that since there were several  -  characters within signature_1, it was splitting the string way before the actual  -  that separates the two. Since both strings are also separated by white spaces, I was able to get around that using following:

"^\w+\s+\|\s+\d+-\d+-\d+\s+\d+:\d+:\d+\s+\w+\s+\|\s+\w+\s+=\s+.+?\|\s+\w+\s+=\s+.+?\|(?<signature_1>.+?)\s\-\s(?<signature_2>.+?)\|"

 

 

0 Karma
Get Updates on the Splunk Community!

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...