topic Re: How to log JSON to Splunk and optimize for spath? in Getting Data In

How to log JSON to Splunk and optimize for spath?

thomasreggi — Fri, 26 Jan 2018 15:59:15 GMT

I am looking to reformat my log output. Right now it's pretty messy and does not follow Splunks parsing format.

What I would like to do is only log a JSON object a string, something like this:

{"time":"2018-01-26 08:17:22.387","uuid":"13hiuh312-213e-134j-sdasj-dsadqweq","file":"main","fn":"load","message":"Hello World"}

Or with a nested "Person" JSON object.

{"time":"2018-01-26 08:17:22.387","uuid":"13hiuh312-213e-134j-sdasj-dsadqweq","file":"main","fn":"load","person":{"name": "thomas reggi", "age":"30"}}

I found in the "Logging best practices" Page that it fields should be all caps and quoted in a totally different format. So I am interested in making the following change.

TIME="2018-01-26 08:17:22.387", UUID="13hiuh312-213e-134j-sdasj-dsadqweq", FILE="main", FN="load", MESSAGE="Hello World"

TIME="2018-01-26 08:17:22.387", UUID="13hiuh312-213e-134j-sdasj-dsadqweq", FILE="main", FN="load", PERSON={"name":"thomas reggi","age":"30"}

I am curious if I need to have the PERSON JSON object at the enclosed in quotes, and if this is valid.

Re: How to log JSON to Splunk and optimize for spath?

mayurr98 — Fri, 26 Jan 2018 16:56:08 GMT

hey i do not think it is a best pratice, rather it would make your search terrible when you want to search for something
Try this run anywhere search

| makeresults | eval raw="{\"time\":\"2018-01-26 08:17:22.387\",\"uuid\":\"13hiuh312-213e-134j-sdasj-dsadqweq\",\"file\":\"main\",\"fn\":\"load\",\"person\":{\"name\": \"thomas reggi\", \"age\":\"30\"}}" | spath input=raw | search fn=load | eval ss="\"load\""  | eval test="load" | search test="load"

suppose if fn="load" then you can not write |search fn=load..if fn=load then you can write both fn=load OR fn="load"

let me know if this helps!

Re: How to log JSON to Splunk and optimize for spath?

niketn — Fri, 26 Jan 2018 17:23:08 GMT

@thomasreggi which "Logging best practices" are you referring too? Can you add detail/link? Is this with respect to the application generating above JSON or Splunk? I think Splunk has no issues with field names in upper case or lower case. Field names should ideally not start with digits or special characters.

Also for the JSON itself to be valid for spath to work automatically, you should have field names also in double quotes as in your first sample JSON. Following is the run anywhere search:

| makeresults 
| eval _raw="{\"TIME\":\"2018-01-26 08:17:22.387\",\"UUID\":\"13hiuh312-213e-134j-sdasj-dsadqweq\",\"FILE\":\"main\",\"FN\":\"load\",\"PERSON\":{\"NAME\": \"thomas reggi\", \"AGE\":\"30\"}}"
| spath

Re: How to log JSON to Splunk and optimize for spath?

thomasreggi — Fri, 26 Jan 2018 21:24:20 GMT

http://dev.splunk.com/view/logging/SP-CAAAFCK