Getting Data In

Help splitting out a JSON file into multiple events

dumdees
Explorer

Hi All,

There are lots of forum topics here on this but I'm really struggling to get my head around it. 

I have the following information in JSON: 

 

 

{
    "4": {
        "state": {
            "on": false,
            "bri": 254,
            "hue": 8418,
            "sat": 140,
            "effect": "none",
            "xy": [
                0.5053,
                0.4152
            ],
            "ct": 454,
            "alert": "select",
            "colormode": "ct",
            "mode": "homeautomation",
            "reachable": false
        },
        "swupdate": {
            "state": "transferring",
            "lastinstall": "2020-03-03T14:19:37"
        },
        "type": "Extended color light",
        "name": "Hue lightstrip plus 1",
        "modelid": "LST002",
        "manufacturername": "Signify Netherlands B.V.",
        "productname": "Hue lightstrip plus",
        "capabilities": {
            "certified": true,
            "control": {
                "mindimlevel": 40,
                "maxlumen": 1600,
                "colorgamuttype": "C",
                "colorgamut": [
                    [
                        0.6915,
                        0.3083
                    ],
                    [
                        0.17,
                        0.7
                    ],
                    [
                        0.1532,
                        0.0475
                    ]
                ],
                "ct": {
                    "min": 153,
                    "max": 500
                }
            },
            "streaming": {
                "renderer": true,
                "proxy": true
            }
        },
        "config": {
            "archetype": "huelightstrip",
            "function": "mixed",
            "direction": "omnidirectional",
            "startup": {
                "mode": "safety",
                "configured": true
            }
        },
        "uniqueid": "00:17:88:01:04:06:ae:3d-0b",
        "swversion": "1.50.2_r30933",
        "swconfigid": "59F2C3A3",
        "productid": "Philips-LST002-1-LedStripsv3"
    },
    "5": {
        "state": {
            "on": false,
            "bri": 144,
            "hue": 7676,
            "sat": 199,
            "effect": "none",
            "xy": [
                0.5016,
                0.4151
            ],
            "ct": 443,
            "alert": "select",
            "colormode": "xy",
            "mode": "homeautomation",
            "reachable": true
        },
        "swupdate": {
            "state": "noupdates",
            "lastinstall": "2021-08-13T13:53:48"
        },
        "type": "Extended color light",
        "name": "Upstairs Hall",
        "modelid": "LCT015",
        "manufacturername": "Signify Netherlands B.V.",
        "productname": "Hue color lamp",
        "capabilities": {
            "certified": true,
            "control": {
                "mindimlevel": 1000,
                "maxlumen": 806,
                "colorgamuttype": "C",
                "colorgamut": [
                    [
                        0.6915,
                        0.3083
                    ],
                    [
                        0.17,
                        0.7
                    ],
                    [
                        0.1532,
                        0.0475
                    ]
                ],
                "ct": {
                    "min": 153,
                    "max": 500
                }
            },
            "streaming": {
                "renderer": true,
                "proxy": true
            }
        },
        "config": {
            "archetype": "sultanbulb",
            "function": "mixed",
            "direction": "omnidirectional",
            "startup": {
                "mode": "safety",
                "configured": true
            }
        },
        "uniqueid": "00:17:88:01:04:ff:49:53-0b",
        "swversion": "1.88.1",
        "swconfigid": "76B74E79",
        "productid": "Philips-LCT015-1-A19ECLv5"
    },

 

 


I am wanting information for "4" and "5" to be ingested as separate events at index time. I understand that one could use regex to filter this properly, but honestly I'm struggling to wrap my head around how. 

Any help would be massively appreciated.

Many Thanks,

John

Labels (1)
0 Karma
1 Solution

dumdees
Explorer

Absolutely amazing. Can't thank you enough!

I really need to read up more on how Splunk uses regex to split event data like this. 

View solution in original post

0 Karma

somesoni2
Revered Legend

Give this a try

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?=\s+\"\d+\":)
NO_BINARY_CHECK=true
CHARSET=UTF-8
disabled=false
SEDCMD-removeHeader=s/^\{$//
SEDCMD-addHeader=s/^\s+\"/{"/
SEDCMD-addFooter=s/\},$/}\n}/

You'll have to add timestamp parsing config as well (use timestamp on the event OR use current time)

dumdees
Explorer

That's excellent thank you! Got that working, however all the field names have the number in front of them... Any idea how I'd get around this?

 

Screenshot 2022-06-01 at 21.53.42.png

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@dumdees - I've not tested it but try this LINE_BREAKER in the staging/testing environment.

(,*{*[\r\n]+\s+\"\d+\":\s*)

 

I hope this helps!!!

dumdees
Explorer

Absolutely amazing. Can't thank you enough!

I really need to read up more on how Splunk uses regex to split event data like this. 

0 Karma
Get Updates on the Splunk Community!

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2025 SplunkTrust is officially open! If you ...

Splunk Answers Content Calendar, June Edition II

Get ready to dive into Splunk Dashboard panels this week! We'll be tackling common questions around ...

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...