Log with JSON and analyze with spath

[opticsplanet]

So, I've read an article about Logging best practices: http://dev.splunk.com/view/SP-CAAADP6

One of the recommendations is to write JSON data and use spath to analyze it. I tried that, and I am not getting any results. Here is the event I'm trying to parse:

{"utma":"23106789.409091805.1340981149.1340981149.1340981149.1","__utmb":"23106789.4.9.1340982149854","__utmc":"23106789","__utmz":"23106789.1340981149.1.1.utmgclid=CL_38obY87ACFdFsTAodxH1OuQ|utmccn=(not set)|utmcmd=(not set)|utmctr=vortexoptics.com"}

. . . | spath - gives me no additional fields.
. . . | spath output=var path=utma - gives me an empty var variable on the output.

What am I missing here?

EDIT: Updated the event to not contain double underscores for variable names, on a suggestion that those are hidden. Still same result.

[afirth]

It can actually, just has to be a field. In your case, you might try spath input=_raw. which is the default, not sure why it's not working

e.g.
Event:

blah=monkeys json_field={"foo":"5022","bar":"d062"} quack=typewriter

search monkeys | spath input=json_field

Will give you foo=5022 and bar=d062

So you could write a regex like

rex field=_raw "(?<json_field>{.+})"

^might need {.+}

Which should throw everything from the first { to the last } into field json_field

good luck

edit: http://splunk-base.splunk.com/answers/61235/kv_modejson-with-combined-json-textual-loglines

same thing

[opticsplanet · Accepted solution]

Ok, figured out the issue. Splunk won't parse out JSON unless the WHOLE event is a JSON object. Or probably starts with JSON code. Otherwise - spath will not work.

[lguinn2]

Actually, spath should work on a partial event. You need to extract the part of the event that is JSON into a field (you can use rex) and then ask spath to parse the field.

yoursearchhere
| rex "(?<json_input>regex to create new field)"
| spath input=json_input

might work, especially if you were only showing a partial event in your question.

[gkanapathy]

I think your problem is that field names beginning with an underscore (_) character in Splunk are hidden from display by default. You actually could use:

... | spath | eval u=___utma

to copy the value to a displayed/visible field, or otherwise manipulate it.

[opticsplanet]

Removed underscores, still same problem.