Getting Data In

how to extract json fields in a mixed type log outside SPL (spath) in props.conf/transform.conf

Loves-to-Learn

I have a logfile with "|" (pipe) seperated field. So i use transform.conf to seperate those fields.

two of the fields I separate are json. Usually  I use spath in SPL to extract the fields in this json field. The jsonfield is just a payload which is loged in the logfile next to user fields. My problem is that "spath" is not available in datasets, so I need to extract the fields automatically and not in SPL. The problem is, that not the whole file is json, so I think I cannot use KV_MODE=json in props conf.

Is there a way to assing just the two json fields to a transform.conf extract the fields in it?

How can I make the fields in the json field available for datasets root events, because I need to accelerate this dataset.

 

Here an example of one event in the file (they are also nested)

_time | field | field | field | field | field | field | field | field | field |
{\"key\":\"value\",\"key\":{\"key\":[\"value\",\"value\",\"value\",\"value\",\"value\"],\"key\":[\"value\"],\"key\":[\"value\"]},\"key\":\"value\",\"key\":\"value\",\"key\":\"value\",\"key\":\"value\",\"key\":\"value\",\"key\":value,\"key\":value}
|
{\"key\":{\"key\":\"value\",\"key\":\"value\",\"key\":[\"value\"],\"key\":[value],\"key\":\"value\",\"key\":\"value\",\"key\":value,\"key\":{\"key\":value,\"key\":value,\"key\":value,\"key\":value,\"key\":value,\"key\":value,\"color\":value,\"key\":value,\"key\":value}},\"key\":[{\"key\":\"value\",\"key\":\"value\"},{\"key\":\"value\",\"key\":\"value\"}]}

AS transform.conf seperate fields by seperator "|" the jsons are seperated fields why I usually use  | spath input=json_field

 

thansk for your support

Labels (2)
0 Karma