Splunk Search

How to extract key-value pairs from log and visualize them

arnavkumarsaxen
Explorer

My logs are in the format:

 

My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121

 

I currently use query:

index="myindex" "My-Application Log:" | extract pairdelim=",  " kvdelim="= " | table Some-Key  SomeOtherKey SomeOtherKey2 "Some Different Key" "Another Key"

 

It is able to extract events however the table is filled with blank/null values.

 

How can i visualise the data if i have this format of logs.

I have to group by Some-key.

Example visualization should be grouped basis Some-key

Thanks in advance.

0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121"
``` use rex to extract kv pairs ```
| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)"
``` streamstats to tag each event so that the fields can be gathered together ```
| streamstats count as _row
``` expand the kv pairs to separate events ``` 
| mvexpand namevalue
``` split names and values ```
| rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)"
| fields - namevalue
``` create new fields for each name with corresponding value ```
| eval {_name}=_value
``` gather fields back to original events ```
| stats values(_time) as _time values(*) as * by _row

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121"
``` use rex to extract kv pairs ```
| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)"
``` streamstats to tag each event so that the fields can be gathered together ```
| streamstats count as _row
``` expand the kv pairs to separate events ``` 
| mvexpand namevalue
``` split names and values ```
| rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)"
| fields - namevalue
``` create new fields for each name with corresponding value ```
| eval {_name}=_value
``` gather fields back to original events ```
| stats values(_time) as _time values(*) as * by _row

arnavkumarsaxen
Explorer

Works for me. However, i am new to splunk and i might have a small question.

The value from the last field contains \n" and sometimes \n"}

How to remove this as well.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try it like this

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\n\}]+)"
0 Karma

arnavkumarsaxen
Explorer

arnavkumarsaxen_0-1650366380136.png

Values still coming in this way

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Assuming you don't want anything after (and including) a backslash, try this:

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\\\\]+)(?=\\\\)?"

arnavkumarsaxen
Explorer

Thanks a lot. It worked.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...