topic Re: Get only parsed JSON fields using spath in Splunk Dev

Get only parsed JSON fields using spath

nagar57 — Mon, 08 Feb 2021 08:04:11 GMT

I have a below JSON

Recalibration Stats json : {"modelid" : "30013", "champion_gini" : 0.8274502273019728, "recalibResult" : "CASE I Champion Retained", "challenger_gini" : 0.8013221831674033, "recalibDate" : "2020-05-01"}

Now to get the JSON fields I have to explicitly mention field names using table/fields. My JSON can get different fields from source so I want to get only parsed fields from JSON by not explicitly mentioning their names. Using below query is giving me all the unwanted fields as well.

index = abx sourcetype = gmdevops_rome source="/axp/gnics/orchestra/dev/romedata/logs/model_run_qc.log" "Recalibration Stats json"
| rex field=_raw "Recalibration Stats json : (?<recalib_stats>.+)"
| spath input=recalib_stats
| table *

Re: Get only parsed JSON fields using spath

ITWhisperer — Mon, 08 Feb 2021 08:12:46 GMT

Not sure what the question is? Are you trying to remove the "unwanted" fields without mentioning which fields are wanted?

Re: Get only parsed JSON fields using spath

nagar57 — Mon, 08 Feb 2021 08:22:06 GMT

Yes, I want only parsed fields from JSON. If I use | table * then I want only those fields that are present in JSON. Unwanted fields are date_hour, _raw, _time , date_second etc.

Re: Get only parsed JSON fields using spath

ITWhisperer — Mon, 08 Feb 2021 08:59:18 GMT

Remove everything apart from recalib_stats after the rex, then remove recalib_stats after the spath

index = abx sourcetype = gmdevops_rome source="/axp/gnics/orchestra/dev/romedata/logs/model_run_qc.log" "Recalibration Stats json" | rex field=_raw "Recalibration Stats json : (?<recalib_stats>.+)" | fields recalib_stats | spath input=recalib_stats | fields - recalib_stats | table *

Re: Get only parsed JSON fields using spath

nagar57 — Mon, 08 Feb 2021 09:07:13 GMT

It'll be like:

index = abx sourcetype = gmdevops_rome source="/axp/gnics/orchestra/dev/romedata/logs/model_run_qc.log" "Recalibration Stats json" | rex field=_raw "Recalibration Stats json : (?<recalib_stats>.+)" | fields recalib_stats | spath input=recalib_stats | fields - recalib_stats, _raw, _time | table *

Re: Get only parsed JSON fields using spath

ITWhisperer — Mon, 08 Feb 2021 09:14:26 GMT

Yes, removing _raw and _time as well. The important step is the selecting of recalib_stats field after the rex which removes the bulk of the other fields - the special fields beginning with an underscore have to be remove individually.