topic how to parse json to extract multiple line event ? in Getting Data In

how to parse json to extract multiple line event ?

sfatnass — Thu, 08 Sep 2016 12:59:31 GMT

i have one file json that contain many object like that :

{
    "id": 1,
    "name": "toto",
    "price": 1.50,
    "tags": ["travel", "red"] } 
{
        "id": 2,
        "name": "toto",
        "price": 12,
        "tags": ["home", "green"] }

i need to extract that on two event for the example : how can i use line_breaker on props.conf

thx

Re: how to parse json to extract multiple line event ?

jkat54 — Thu, 08 Sep 2016 13:34:20 GMT

This method will index each field name in the json payload:

[ <SOURCETYPE NAME> ]  
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
CHARSET=AUTO
INDEXED_EXTRACTIONS=json
KV_MODE=none
disabled=false
pulldown_type=true

This would not and would come at a lower performance cost:

[ <SOURCETYPE NAME> ]
CHARSET=AUTO
SHOULD_LINEMERGE=false
disabled=false
LINE_BREAKER=(^){.*"id":

Re: how to parse json to extract multiple line event ?

sfatnass — Thu, 08 Sep 2016 13:40:31 GMT

this is my current configuraiton in the props.conf
[json]

[source::.../mysource...]
sourcetype = json
SHOULD_LINEMERGE = false
TRUNCATE=0
NO_BINARY_CHECK = 1
LINE_BREAKER = ([\r\n]+){

then i need to have something like that :

[json]

[source::.../mysource...]
 SHOULD_LINEMERGE=true
 NO_BINARY_CHECK=true
 CHARSET=AUTO
 INDEXED_EXTRACTIONS=json
 KV_MODE=json
 disabled=false
 pulldown_type=true

it's ok like that, it work very well and the performence is great thx

Re: how to parse json to extract multiple line event ?

jkat54 — Thu, 08 Sep 2016 13:58:48 GMT

Do you want the fields extracted at index time or search time?

Both examples I gave you worked with your example data so either you didn't reindex the data, didn't put the props in the correct place, or maybe the example data you provided isn't exactly like the data you're ingesting.

Re: how to parse json to extract multiple line event ?

jkat54 — Thu, 08 Sep 2016 14:00:05 GMT

The settings you used would index the fields and would need to be placed on the universal forwarder and indexers. It wouldn't apply to data already ingested either.

Re: how to parse json to extract multiple line event ?

sfatnass — Thu, 08 Sep 2016 18:57:54 GMT

just to extract json for many event like your exemple and extract all field too, because i will use some request and i need to know who the field contain the correct value ^^
but it's ok and thx for your reply

Re: how to parse json to extract multiple line event ?

jkat54 — Thu, 08 Sep 2016 19:43:37 GMT

great, just so you know the INDEXED_EXTRACTIONS will consume more disk space and does require more CPU on the indexers/forwarders

Re: how to parse json to extract multiple line event ?

sfatnass — Thu, 08 Sep 2016 20:51:11 GMT

ok but it's more performent no? the objectif of my project is to play more speed ^^

Re: how to parse json to extract multiple line event ?

jkat54 — Thu, 08 Sep 2016 21:34:46 GMT

It can be faster when you're searching for the fields involved yes.