Getting Data In

How to Split the below json file into multi events

Inayath_khan
Path Finder

Hi Folks,

Kindly help me to figure out dividing the below logs into each events.

     {
        "SecurityGroups": [
            {
                "Description": "default VPC security group",
                "GroupId": "abcd",
                "GroupName": "default",
                "IpPermissions": [
                    {
                        "FromPort": 23,
                        "IpProtocol": "tcp",
                        "IpRanges": [
                            {
                                "CidrIp": "0.0.0.0/0"
                            }
                        ],
                        "Ipv6Ranges": [
                            {
                                "CidrIpv6": "::/0"
                            }
                        ],
                        "PrefixListIds": [],
                        "ToPort": 22,
                        "UserIdGroupPairs": []
                    },
                    {
                        "FromPort": 443,
                        "IpProtocol": "tcp",
                        "IpRanges": [
                            {
                                "CidrIp": "0.0.0.0/0",
                                "Description": "For logs"
                            }
                        ],
                        "Ipv6Ranges": [
                            {
                                "CidrIpv6": "::/0",
                                "Description": "For logs"
                            }
                        ],
                        "PrefixListIds": [],
                        "ToPort": 443,
                        "UserIdGroupPairs": []
                    }
                ],
                "IpPermissionsEgress": [],
                "OwnerId": "##########",
                "VpcId": "vpc-#############"
            },
            {
                "Description": "launch-wizard-10 created 2019-03-27T18:10:47.749-07:00",
                "GroupId": "sg-#############",
                "GroupName": "launch-wizard-10",
                "IpPermissions": [
                    {
                        "FromPort": 22,
                        "IpProtocol": "tcp",
                        "IpRanges": [
                            {
                                "CidrIp": "0.0.0.0/0"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "ToPort": 22,
                        "UserIdGroupPairs": []
                    }
                ],
                "IpPermissionsEgress": [
                    {
                        "IpProtocol": "-1",
                        "IpRanges": [
                            {
                                "CidrIp": "0.0.0.0/0"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "UserIdGroupPairs": []
                    }
                ],
                "OwnerId": "439086027177",
                "VpcId": "vpc-01d67bc62cd3d2d62"
            },
            {
                "Description": "Security Group",
                "GroupId": "##############",
                "GroupName": "prd-#################",
                "IpPermissions": [
                    {
                        "FromPort": 8093,
                        "IpProtocol": "tcp",
                        "IpRanges": [
                            {
                                "CidrIp": "10.0.0.1/25",
                                "Description": "############"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "ToPort": 9093,
                        "UserIdGroupPairs": [
                            {
                                "Description": "########## ",
                                "GroupId": "##########",
                                "UserId": "#############"
                            }
                        ]
                    },
                    {
                        "FromPort": 9090,
                        "IpProtocol": "tcp",
                        "IpRanges": [
                            {
                                "CidrIp": "10.0.0.1/19",
                                "Description": "############"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "ToPort": 3899,
                        "UserIdGroupPairs": [
                            {
                                "Description": "######### ",
                                "GroupId": "############",
                                "UserId": "##############"
                            }
                        ]
                    },
                    {
                        "FromPort": 8080,
                        "IpProtocol": "tcp",
                        "IpRanges": [
                            {
                                "CidrIp": "10.0.0.1/23",
                                "Description": "#########"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "ToPort": 443,
                        "UserIdGroupPairs": [
                            {
                                "Description": "HTTPS port",
                                "GroupId": "#############",
                                "UserId": "##############"
                            }
                        ]
                    }
                ],
                "IpPermissionsEgress": [
                    {
                        "IpProtocol": "-1",
                        "IpRanges": [
                            {
                                "CidrIp": "0.0.0.0/0"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "UserIdGroupPairs": []
                    }
                ],
                "OwnerId": "439086027177",
                "Tags": [
                    {
                        "Key": "####",
                        "Value": "#######"
                    },
                    {
                        "Key": "#########",
                        "Value": "################"
                    },
                    {
                        "Key": "####",
                        "Value": "#####"
                    },
                    {
                        "Key": "Name",
                        "Value": "############"
                    }
                ],
                "VpcId": "vpc-#################"    
                }
           ]
   }

Awaiting for a quick response . Kindly reply on the post.

I tried the below method but failed at writing the REGEX.

https://answers.splunk.com/answers/642050/how-to-split-the-following-json-into-different-eve.html

Tags (2)
0 Karma
1 Solution

martynoconnor
Communicator

The lack of a timestamp is an issue - cause if you're reading all of these from a single JSON, you'll have Splunk thinking everything happened at once, at the point of ingest.

Anyway, you can break the JSON into events using BREAK_ONLY_BEFORE = \"Description\":\s+\"[\w\s]+\" in your sourcetype definition and by disabling INDEXED_EXTRACTIONS = json, but then by telling Splunk to ignore its normal parsing of JSON, you lose all of the extracted fields, and you'll need to use EXTRACT-blah in props to pull things out.

View solution in original post

0 Karma

martynoconnor
Communicator

The lack of a timestamp is an issue - cause if you're reading all of these from a single JSON, you'll have Splunk thinking everything happened at once, at the point of ingest.

Anyway, you can break the JSON into events using BREAK_ONLY_BEFORE = \"Description\":\s+\"[\w\s]+\" in your sourcetype definition and by disabling INDEXED_EXTRACTIONS = json, but then by telling Splunk to ignore its normal parsing of JSON, you lose all of the extracted fields, and you'll need to use EXTRACT-blah in props to pull things out.

0 Karma

martynoconnor
Communicator

How exactly do you want it split? What within the JSON posted do you consider an event breaker?

0 Karma
Get Updates on the Splunk Community!

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...

Splunk MCP & Agentic AI: Machine Data Without Limits

Discover how the Splunk Model Context Protocol (MCP) Server can revolutionize the way your organization uses ...