The "level" field is being automatically added by splunk, how to we ask splunk to extract log level from my json message ?

[kishoresanke]

The "level" field is being automatically added by splunk, how to we ask splunk to extract log level from my json message ?

[micahkemp]

Actually I think you can do this. The previous answer's explanation is valid, but using it you can do this:

props.conf:

[<sourcetype>]
SHOULD_LINEMERGE = false
REPORT-embedded_message = embedded_message
KV_MODE = json

props.conf:

[embedded_message]
SOURCE_KEY = message
REGEX = "(?<_KEY_1>[^:]+)":\s*"(?<_VAL_1>.*?)"

In action:

[kishoresanke]

Thanks a lot again for a detailed answer. I will try this today . quick question - why do we have both KV_MODE = none as well as KV_MODE = json in the sourcetype section ?

[micahkemp]

Artifact of previous attempt. 🙂 WIll fix in answer.

[micahkemp]

The reason splunk isn't automatically extracting this out is (probably) because the message field is JSON embedded in other JSON, as opposed to a sub-structure of the same JSON. I'm willing to bet the message field contains escaped quotes if you view the raw event. Here's a run-anywhere example of what I mean:

| makeresults | eval _raw="{\"message\": \"{\\\"level\\\": \\\"0\\\"}\"}" | spath

Gives a sample _raw of:

{"message": "{\"level\": \"0\"}"}

And a message value of:

{"level": "0"}

Which you could run spath against and get the results you want:

| makeresults | eval _raw="{\"message\": \"{\\\"level\\\": \\\"0\\\"}\"}" | spath | spath input=message

If the message field was part of the actual JSON structure (instead of JSON embedded in other JSON), _raw would look more like this:

{"message": {"level": "0"}}

Which would parse properly into message.level. The run-anywhere search to show this in action is:

| makeresults | eval _raw="{\"message\": {\"level\": \"0\"}}" | spath

So how do you make this "just work" without having to run a separate spath each time you search? I don't think there is a way. Even if you knew that you only cared about the message field and nothing else and you used SEDCMD to rewrite _raw when you index the data, it would still contain escaped quotes that are going to prevent JSON parsing from being successful.

[kishoresanke]

Thanks for detailed explanation. But what bugs me is, the "req_id" field is also escaped within the message field, but somehow splunk has automatically extracted this out. Then why not the "level" ?

[harsmarvania57]

Are you able to see automatically extracted message.level field on left hand side? If yes then it will give you value INFO

[kishoresanke]

no, its not automatically extracted.

[cmerriman]

have you tried spath?
http://docs.splunk.com/Documentation/Splunk/6.3.3/SearchReference/spath

[kishoresanke]

i did try spath, but wouldn't it be easier if splunk extracts the field automatically rather than i write spath command to extract the field everytime i want to search.