Splunk Search

How to split JSON into multiple events using regex?

Zamoraw
New Member

I am currently trying to split my json into multiple events at index time into Splunk. Although when I do this it breaks each line into multiple events. I am not good with regex, so I tried using the regex from the answer here
https://answers.splunk.com/answers/289520/how-to-split-a-json-array-into-multiple-events-wit.html
The answer is exactly how I want my output to be.
Heres my props.conf and my sample json
1.

[jsonsourcetype]
SHOULD_LINEMERGE = FALSE
LINE_BREAKER = ((?<!")\},|[\r\n]+)
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
SEDCMD-remove_footer = s/\]\s+\}//g
TIME_PREFIX = \"CreatedDate\":\s+\"

2.

{
"records": [
    {
        "field1": "923893829413",
        "CreatedDate": "2018-08-10T06:24:35.000+0000",
        "Id": "a8928371DL0",
        "attributes": {
            "type": "F",
            "url": "/something/etc/test"
        }
    },               
  {
        "field1": "923829323829413",
        "Id": "a8921238371DL01",
        "attributes": {
            "type": "TF",
            "url": "urlHere"
        }
    }          
]
}
0 Karma
1 Solution

amiftah
Communicator

Try this:

BREAK_ONLY_BEFORE = (\[\s+\{)
MUST_BREAK_AFTER = (\},|\}\s+\])
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
SEDCMD-remove_footer = s/\]\s+\}//g

View solution in original post

0 Karma

amiftah
Communicator

Did you try it?
I have two separate events, unless if it's not what you want..

alt text

0 Karma

Zamoraw
New Member

Yes, I tried it but my output was one event. This is exactly what I need. But for some reason Its not working on mine.

0 Karma

amiftah
Communicator

Ok, here's the source type I used for this output:

[test22]
BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE
DATETIME_CONFIG =
MUST_BREAK_AFTER = (\},|\}\s+\])
NO_BINARY_CHECK = true
SEDCMD-remove_footer = s/\]\s+\}//g
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
TIME_PREFIX = \"CreatedDate\":\s+\"
category = Custom
pulldown_type = true

Try to reindex your file with that same source type

Zamoraw
New Member

Thank you this works and I understand most of it except that first line BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE. Could you explain that a bit?

0 Karma

amiftah
Communicator

Try this:

BREAK_ONLY_BEFORE = (\[\s+\{)
MUST_BREAK_AFTER = (\},|\}\s+\])
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
SEDCMD-remove_footer = s/\]\s+\}//g
0 Karma

Zamoraw
New Member

This just seems to put it back into one event. I need multiple events.

0 Karma

Zamoraw
New Member

Awesome, Thanks! After adding this with the comment you posted in the other answer it worked great! Thank you.

0 Karma
Get Updates on the Splunk Community!

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...