topic Re: Log with JSON and analyze with spath in Splunk Search

Log with JSON and analyze with spath

opticsplanet — Tue, 17 Jul 2012 00:38:05 GMT

So, I've read an article about Logging best practices: http://dev.splunk.com/view/SP-CAAADP6

One of the recommendations is to write JSON data and use spath to analyze it. I tried that, and I am not getting any results. Here is the event I'm trying to parse:

{"utma":"23106789.409091805.1340981149.1340981149.1340981149.1","__utmb":"23106789.4.9.1340982149854","__utmc":"23106789","__utmz":"23106789.1340981149.1.1.utmgclid=CL_38obY87ACFdFsTAodxH1OuQ|utmccn=(not set)|utmcmd=(not set)|utmctr=vortexoptics.com"}

. . . | spath - gives me no additional fields.
. . . | spath output=var path=utma - gives me an empty var variable on the output.

What am I missing here?

EDIT: Updated the event to not contain double underscores for variable names, on a suggestion that those are hidden. Still same result.

Re: Log with JSON and analyze with spath

gkanapathy — Tue, 17 Jul 2012 01:31:05 GMT

I think your problem is that field names beginning with an underscore (_) character in Splunk are hidden from display by default. You actually could use:

... | spath | eval u=___utma

to copy the value to a displayed/visible field, or otherwise manipulate it.

Re: Log with JSON and analyze with spath

opticsplanet — Tue, 17 Jul 2012 16:39:53 GMT

Removed underscores, still same problem.

Re: Log with JSON and analyze with spath

opticsplanet — Thu, 26 Jul 2012 22:34:34 GMT

Ok, figured out the issue. Splunk won't parse out JSON unless the WHOLE event is a JSON object. Or probably starts with JSON code. Otherwise - spath will not work.

Re: Log with JSON and analyze with spath

afirth — Wed, 17 Oct 2012 21:37:51 GMT

It can actually, just has to be a field. In your case, you might try spath input=_raw. which is the default, not sure why it's not working

e.g.
Event:

blah=monkeys json_field={"foo":"5022","bar":"d062"} quack=typewriter

search monkeys | spath input=json_field

Will give you foo=5022 and bar=d062

So you could write a regex like

rex field=_raw "(?<json_field>{.+})"

^might need {.+}

Which should throw everything from the first { to the last } into field json_field

good luck

edit: http://splunk-base.splunk.com/answers/61235/kv_modejson-with-combined-json-textual-loglines

same thing

Re: Log with JSON and analyze with spath

lguinn2 — Tue, 12 Feb 2013 08:22:15 GMT

Actually, spath should work on a partial event. You need to extract the part of the event that is JSON into a field (you can use rex) and then ask spath to parse the field.

yoursearchhere | rex "(?<json_input>regex to create new field)" | spath input=json_input

might work, especially if you were only showing a partial event in your question.