topic Re: How to split JSON into multiple events using regex? in Splunk Search

How to split JSON into multiple events using regex?

Zamoraw — Tue, 14 Aug 2018 21:25:34 GMT

I am currently trying to split my json into multiple events at index time into Splunk. Although when I do this it breaks each line into multiple events. I am not good with regex, so I tried using the regex from the answer here
https://answers.splunk.com/answers/289520/how-to-split-a-json-array-into-multiple-events-wit.html
The answer is exactly how I want my output to be.
Heres my props.conf and my sample json
1.

[jsonsourcetype]
SHOULD_LINEMERGE = FALSE
LINE_BREAKER = ((?<!")\},|[\r\n]+)
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
SEDCMD-remove_footer = s/\]\s+\}//g
TIME_PREFIX = \"CreatedDate\":\s+\"

{
"records": [
    {
        "field1": "923893829413",
        "CreatedDate": "2018-08-10T06:24:35.000+0000",
        "Id": "a8928371DL0",
        "attributes": {
            "type": "F",
            "url": "/something/etc/test"
        }
    },               
  {
        "field1": "923829323829413",
        "Id": "a8921238371DL01",
        "attributes": {
            "type": "TF",
            "url": "urlHere"
        }
    }          
]
}

Re: How to split JSON into multiple events using regex?

amiftah — Tue, 14 Aug 2018 21:57:51 GMT

Try this:

BREAK_ONLY_BEFORE = (\[\s+\{)
MUST_BREAK_AFTER = (\},|\}\s+\])
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
SEDCMD-remove_footer = s/\]\s+\}//g

Re: How to split JSON into multiple events using regex?

Zamoraw — Tue, 14 Aug 2018 22:16:07 GMT

This just seems to put it back into one event. I need multiple events.

Re: How to split JSON into multiple events using regex?

amiftah — Wed, 15 Aug 2018 00:10:53 GMT

Did you try it?
I have two separate events, unless if it's not what you want..

Re: How to split JSON into multiple events using regex?

Zamoraw — Wed, 15 Aug 2018 00:15:00 GMT

Yes, I tried it but my output was one event. This is exactly what I need. But for some reason Its not working on mine.

Re: How to split JSON into multiple events using regex?

amiftah — Wed, 15 Aug 2018 00:17:40 GMT

Ok, here's the source type I used for this output:

[test22]
BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE
DATETIME_CONFIG =
MUST_BREAK_AFTER = (\},|\}\s+\])
NO_BINARY_CHECK = true
SEDCMD-remove_footer = s/\]\s+\}//g
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
TIME_PREFIX = \"CreatedDate\":\s+\"
category = Custom
pulldown_type = true

Try to reindex your file with that same source type

Re: How to split JSON into multiple events using regex?

Zamoraw — Wed, 15 Aug 2018 16:04:39 GMT

Awesome, Thanks! After adding this with the comment you posted in the other answer it worked great! Thank you.

Re: How to split JSON into multiple events using regex?

Zamoraw — Tue, 29 Sep 2020 20:52:33 GMT

Thank you this works and I understand most of it except that first line BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE. Could you explain that a bit?