topic break a json log in Getting Data In

break a json log

gcusello — Fri, 11 Sep 2020 11:00:21 GMT

Hi at all,

I have a json log that in a single json contains many events:

{"response":{"caseEvents":[{"eventDetails":{"eventDescription":"SCT","eventId":"TRX8551","eventTime":"2020-06-24T13:21:00.664+00:00","eventType":"PAYMENT"}},{"eventDetails":{"eventDescription":"SCT","eventId":"TRX8552","eventTime":"2020-06-24T13:21:01.664+00:00","eventType":"PAYMENT"}}}]}

in the same json I have many eventDetails section (here only two with few fields, but they are many more).

I tried to use:

indexed_extractions=JSON

and

LINE_BREAKER = \{\"eventDetails\"

but it still remains one.

How can I approach the problem?

Ciao.

Giuseppe

Re: break a json log

thambisetty — Fri, 11 Sep 2020 11:45:17 GMT

would there be any option to split them before indexing to Splunk? instead of depending on props.conf , is it possible to make changes in modular inputs if you are using one?

Re: break a json log

gcusello — Fri, 11 Sep 2020 12:53:55 GMT

Hi @thambisetty,

I'd like to avoid to pre-elaborate the source with a script before indexing!

I'd like to find a way to split events in Splunk.

Have you another idea?

Ciao.

Giuseppe

Re: break a json log

uwehermann — Fri, 11 Sep 2020 13:23:29 GMT

Well, depending on the formatting of the json log files, you at least need the following in props.conf:

SHOULD_LINEMERGE = false

If the data is not prettyprinted, the following should help:

LINE_BREAKER= \}(,)\{

If the data is prettyprinted, the following may be used to allow for whitespaces between the comma and the curly brace:

LINE_BREAKER = \}(,\s*)\{

Re: break a json log

thambisetty — Sun, 13 Sep 2020 06:38:48 GMT

@gcusello

please have a look at below

[ __auto__learned__ ] SHOULD_LINEMERGE=true LINE_BREAKER=(\{\"eventDetails\"\:) NO_BINARY_CHECK=true SEDCMD-tailchars=s/(.*\"\}{1}).*/\1/g

Added SEDCMD class to remove unwanted characters at end of the line.

you need to deal with pre-text which is in first event.