topic Re: How to extract key-value pairs from log and visualize them in Splunk Search

How to extract key-value pairs from log and visualize them

arnavkumarsaxen — Tue, 19 Apr 2022 08:35:37 GMT

My logs are in the format:

My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121

I currently use query:

index="myindex" "My-Application Log:" | extract pairdelim=", " kvdelim="= " | table Some-Key SomeOtherKey SomeOtherKey2 "Some Different Key" "Another Key"

It is able to extract events however the table is filled with blank/null values.

How can i visualise the data if i have this format of logs.

I have to group by Some-key.

Example visualization should be grouped basis Some-key

Thanks in advance.

Re: How to extract key-value pairs from log and visualize them

ITWhisperer — Tue, 19 Apr 2022 08:55:32 GMT

| makeresults | eval _raw="My-Application Log: Some-Key= 99, SomeOtherKey= 231, SomeOtherKey2= 1231, Some Different Key= 0, Another Key= 121" ``` use rex to extract kv pairs ``` | rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,]+)" ``` streamstats to tag each event so that the fields can be gathered together ``` | streamstats count as _row ``` expand the kv pairs to separate events ``` | mvexpand namevalue ``` split names and values ``` | rex field=namevalue "(?<_name>[^=]+)\=\s(?<_value>.*)" | fields - namevalue ``` create new fields for each name with corresponding value ``` | eval {_name}=_value ``` gather fields back to original events ``` | stats values(_time) as _time values(*) as * by _row

Re: How to extract key-value pairs from log and visualize them

arnavkumarsaxen — Tue, 19 Apr 2022 09:59:06 GMT

Works for me. However, i am new to splunk and i might have a small question.

The value from the last field contains \n" and sometimes \n"}

How to remove this as well.

Re: How to extract key-value pairs from log and visualize them

ITWhisperer — Tue, 19 Apr 2022 10:48:09 GMT

Try it like this

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\n\}]+)"

Re: How to extract key-value pairs from log and visualize them

arnavkumarsaxen — Tue, 19 Apr 2022 11:06:36 GMT

Values still coming in this way

Re: How to extract key-value pairs from log and visualize them

ITWhisperer — Tue, 19 Apr 2022 11:35:26 GMT

Assuming you don't want anything after (and including) a backslash, try this:

| rex max_match=0 "(?<namevalue>\w[\w\s\-]+=\s[^,\\\\]+)(?=\\\\)?"

Re: How to extract key-value pairs from log and visualize them

arnavkumarsaxen — Tue, 19 Apr 2022 12:10:53 GMT

Thanks a lot. It worked.