Is it possible to parse an extracted field as json if the whole log line isn't json?

[stevennoble]

If I have a line of my logs that look something like

[2013-10-18 23:36:50.785476] {"message":"some message", "headers": {"a": 1, "b": 2}}

is there a way I can use splunk's regex extraction to separate the timestamp from the json, then use splunk's json extraction to to extract fields from the json?

[kamermans]

You can remove the string leader using any the following methods:

1. Remove leader by pretending it's a line breaker
   
LINE_BREAKER=((:?^|\n).+?){
SHOULD_LINEMERGE=false


2. Removing the leader with SEDCMD:
   
SEDCMD-StripHeader=s/^[^{]+//


3. Removing the leader via a transform on _raw:
   
; in transforms.conf:
[StripSyslog]
REGEX = ^[^{]+(.*)$
FORMAT = $1
DEST_KEY = _raw


; in props.conf:
TRANSFORMS-StripSyslog = StripSyslog


All these methods will work with KV_MODE=json, but note that currently they will not work with INDEXED_EXTRACTIONS=json.

[prees]

Second what @wsnyder2 said. Is it possible to do this and still retain the info from the header?

EDIT: or is it possible to pre-transform them based on the header then apply this transform?

[wsnyder2]

Can I do this and keep my header?

[_d_ · Accepted solution]

You can't use KV_MODE=json in props.conf because in its current state the event is not fully json. Additionally you can't extract the rest of the messages and then use the same setting on it (again, from props.conf). However, you can do it inline with spath.

Extract the whole json message in a field called, say, my_field, then use spath:

...| spath input=my_field

[wsnyder2]

This is a bummer ... We have jason as part of an event. Would love to have field extraction happen under the hood, e.g. in props.conf or transforms.conf.
Seems like there is a way based on postings in this thread. (?)

[prees]

Was any one able to do this? as originally described, ie. without using spath?

[_d_]

Inline in a search. You can follow the ..|spath input=my_field with any other search commands. More details and examples here: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath

[stevennoble]

This is something that I'd use in a search? Or can this be applied to a log source?