topic JSON events not splitting correctly in Getting Data In

JSON events not splitting correctly

pjohnson1 — Sat, 06 Jun 2020 00:54:02 GMT

I have some data in the following format which does not split correctly.

The events get indexed as one event.

sample data

{"date": "5/8/2020", "time": "7:57:47 AM", "client": "187.45.18.205", "flags": "A", "query": "v1.addthisedge.com"}{"date": "5/8/2020", "time": "7:57:47 AM", "client": "188.35.138.205", "flags": "A", "query": "m.addthis.com"}{"date": "5/8/2020", "time": "7:57:47 AM", "client": "186.95.16.121", "flags": "A", "query": "cloud.acrobat.com"}

props.conf

[monitor:///data/dns/*/*/*/*.json.log]
INDEXED_EXTRACTIONS = json
KV_MODE = none

Re: JSON events not splitting correctly

to4kawa — Fri, 08 May 2020 02:25:12 GMT

your JSON is not valid.
, is missing between objects( {\"date...} )
Both INDEXED_EXTRACTIONS and KV_MODE can't work.

Re: JSON events not splitting correctly

to4kawa — Fri, 08 May 2020 02:30:48 GMT

[monitor:///data/dns/*/*/*/*.json.log]
SHOULD_LINEMERGE = false
LINE_BREAKER = }(\s*)
NO_BINARY_CHECK = true

Well, if you cut it off, there's no problem.

Re: JSON events not splitting correctly

pjohnson1 — Fri, 08 May 2020 05:30:12 GMT

Thank you for answering. I fixed up the json output but still have an issue.

 [{"date": "5/8/2020", "time": "7:57:47 AM", "client": "187.45.18.205", "flags": "A", "query": "v1.addthisedge.com"},{"date": "5/8/2020", "time": "7:57:47 AM", "client": "188.35.138.205", "flags": "A", "query": "m.addthis.com"},{"date": "5/8/2020", "time": "7:57:47 AM", "client": "186.95.16.121", "flags": "A", "query": "cloud.acrobat.com"}]

Re: JSON events not splitting correctly

to4kawa — Fri, 08 May 2020 05:39:33 GMT

I hope you look my answer before amended.

your json is valid now.

 [monitor:///data/dns/*/*/*/*.json.log]
 INDEXED_EXTRACTIONS = json
 KV_MODE = none

it works.

Re: JSON events not splitting correctly

pjohnson1 — Fri, 08 May 2020 07:22:17 GMT

Yes, I did. Thank you! 🙂