Splunk Search

How to extract key-value pairs from log and visualize them

arnavkumarsaxen
Explorer

My logs are in the format:

 

My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121

 

I currently use query:

index="myindex" "My-Application Log:" | extract pairdelim=",  " kvdelim="= " | table Some-Key  SomeOtherKey SomeOtherKey2 "Some Different Key" "Another Key"

 

It is able to extract events however the table is filled with blank/null values.

 

How can i visualise the data if i have this format of logs.

I have to group by Some-key.

Example visualization should be grouped basis Some-key

Thanks in advance.

0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121"
``` use rex to extract kv pairs ```
| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)"
``` streamstats to tag each event so that the fields can be gathered together ```
| streamstats count as _row
``` expand the kv pairs to separate events ``` 
| mvexpand namevalue
``` split names and values ```
| rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)"
| fields - namevalue
``` create new fields for each name with corresponding value ```
| eval {_name}=_value
``` gather fields back to original events ```
| stats values(_time) as _time values(*) as * by _row

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121"
``` use rex to extract kv pairs ```
| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)"
``` streamstats to tag each event so that the fields can be gathered together ```
| streamstats count as _row
``` expand the kv pairs to separate events ``` 
| mvexpand namevalue
``` split names and values ```
| rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)"
| fields - namevalue
``` create new fields for each name with corresponding value ```
| eval {_name}=_value
``` gather fields back to original events ```
| stats values(_time) as _time values(*) as * by _row

arnavkumarsaxen
Explorer

Works for me. However, i am new to splunk and i might have a small question.

The value from the last field contains \n" and sometimes \n"}

How to remove this as well.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try it like this

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\n\}]+)"
0 Karma

arnavkumarsaxen
Explorer

arnavkumarsaxen_0-1650366380136.png

Values still coming in this way

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Assuming you don't want anything after (and including) a backslash, try this:

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\\\\]+)(?=\\\\)?"

arnavkumarsaxen
Explorer

Thanks a lot. It worked.

0 Karma
Get Updates on the Splunk Community!

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...

Reminder! Splunk Love Promo: $25 Visa Gift Card for Your Honest SOAR Review With ...

We recently launched our first Splunk Love Special, and it's gone phenomenally well, so we're doing it again, ...