How can I extract all fields from a log file including json definiton?

[florianduhme]

I have a log file which entries/lines look like this:

12:17:35.4641 Info {"message":"TestKevin execution ended","level":"Information","logType":"Default","timeStamp":"2018-09-19T12:17:35.4641435+00:00","fingerprint":"2ee56795-e30b-4c98-b6cc-166249d18375","windowsIdentity":"RPA1\\Robotics","machineName":"RPA1","processName":"TestKevin","processVersion":"1.0.6828.23354","fileName":"Main_Alf_Production","jobId":"d415160d-8bfd-4374-b0c4-b03a35316b79","robotName":"ROBOTICS","totalExecutionTimeInSeconds":34,"totalExecutionTime":"00:00:34"}

First of all, I want to extract the time(12:17:35.4641), the Status (Info), and all of the key-value pairs included in the json definition. It would be great to do this at indexing time, so I don't have to include this logic within my search. Any Ideas?

[joebisesi]

So is your example the raw data?

[florianduhme]

Exactly, the example data above would be the _raw data.

But what I could do is to split up the fields "Time" "Status" and the JSON definition into 3 different fields (as explained in the comment to your first answer), if that would help. So we would just need to split up the key-value pairs of the field with the JSON definition.

[joebisesi]

If the data is all JSON, then you need this in your props.conf: KV_MODE=json

http://docs.splunk.com/Documentation/Splunk/7.1.3/Admin/Propsconf

[florianduhme]

I guess that's exactly my problem, that not all of my data is JSON. The first two fields are not part of the JSON definition and therefore this option does not seem to have any impact on the indexed data.

My workaround is to write the JSON to a single field (so I have 3 fields: "Time", "Status", "JSON definition") and the use the "spath" command to expand the Key-Value pairs.

index=<someindex> | spath input=jsonData 

jsonData is the field with the complete JSON definition.

But it would be nice to do this at indexing time. Any other suggestions?