topic Re: Extract key value pairs from json file using regex in Splunk Search

How do I extract key value pairs from json file using regex?

anooshac — Fri, 29 Jul 2022 15:25:27 GMT

Hi all, I have a sample json file like this.

{ "Project Name" : "abc", "Project Group":"A", "Unit":"B", "groups_data":[{ "a":"32.064453125", "b":"5.451171875", "c":"0.3349609375", "d":"0.181640625", "e":"4.58203125", "f":"81.1611328125"}] }

I want to plot a pie chart for the key value pairs present in the groups_data. I tried extracting the data using this query.

myindex sourcetype="_json"| rex field=_raw "\"group_data\":\[\{\"(?<component>[^/]*)\":"\"(?<Value>\d+)\"\}\]| eval tmp = mvzip(component,Value) |mvexpand tmp |eval component=mvindex(split(tmp,","),0) |eval Value=mvindex(split(tmp,","),1)|chart values(Value) by component

I am not able to pie chart. It says tmp does not exist.Can anyone tell me is there anything wrong in the regex part? Something i missed anywhere?

Re: Extract key value pairs from json file using regex

ITWhisperer — Fri, 29 Jul 2022 07:32:09 GMT

Try extracting the groups_data (group_data?) with spath and then use max_match=0 for multiple extracts. Also, use a match string that matches your (example) data

myindex sourcetype="_json" | spath output=groups_data groups_data{} | rex field=groups_data max_match=0 "\"(?<component>[^\"]+)\":\"(?<Value>[\d\.]+)\"" | eval tmp = mvzip(component,Value) | mvexpand tmp | eval component=mvindex(split(tmp,","),0) | eval Value=mvindex(split(tmp,","),1) | chart values(Value) by component

Re: Extract key value pairs from json file using regex

anooshac — Fri, 29 Jul 2022 09:31:32 GMT

Thank you so much it is working.. Is it mandatory to use spath while using regex?

Also some of the keys have comma because of key is splitting, how can i avoid this? How to escape comma present in key?

Re: Extract key value pairs from json file using regex

ITWhisperer — Fri, 29 Jul 2022 09:53:59 GMT

No, spath is not mandatory for use with rex, you could have extracted the groups_data in to a separate field with rex, then used another rex to extract the components and value pairs - the key here is the max_match=0 to apply the extract pattern multiple times.

Can you give an example of where this is failing (due to commas)?

Re: Extract key value pairs from json file using regex

anooshac — Mon, 01 Aug 2022 05:01:19 GMT

Okay.. Thanks alot for the information.

There are some values like below.

"x,y":"32.064453125

this will be extracted as x and y as key and values. How can i extract "x,y" as a key?

Re: Extract key value pairs from json file using regex

ITWhisperer — Mon, 01 Aug 2022 05:12:16 GMT

Extract the pairs as tmp and split using the colon then trim the double-quotes

myindex sourcetype="_json" | spath output=groups_data groups_data{} | rex field=groups_data max_match=0 "(?<tmp>\"[^\"]+\":\"[\d\.]+\")" | mvexpand tmp | eval component=trim(mvindex(split(tmp,":"),0),"\"") | eval Value=trim(mvindex(split(tmp,":"),1),"\"") | chart values(Value) by component

Re: Extract key value pairs from json file using regex

anooshac — Mon, 01 Aug 2022 05:26:03 GMT

Thank you so much for the help! It is working fine now..

Re: Extract key value pairs from json file using regex

jotne — Mon, 01 Aug 2022 07:54:45 GMT

You could also extract component Value using regex.

| eval component=trim(mvindex(split(tmp,":"),0),"\"") | eval Value=trim(mvindex(split(tmp,":"),1),"\"")

| rex field=tmp "\"(?<component>[^\"]+)\":\"(?<Value>[^\"]+)\""