Getting Data In

Help splitting out a JSON file into multiple events

dumdees
Explorer

Hi All,

There are lots of forum topics here on this but I'm really struggling to get my head around it. 

I have the following information in JSON: 

 

 

{
    "4": {
        "state": {
            "on": false,
            "bri": 254,
            "hue": 8418,
            "sat": 140,
            "effect": "none",
            "xy": [
                0.5053,
                0.4152
            ],
            "ct": 454,
            "alert": "select",
            "colormode": "ct",
            "mode": "homeautomation",
            "reachable": false
        },
        "swupdate": {
            "state": "transferring",
            "lastinstall": "2020-03-03T14:19:37"
        },
        "type": "Extended color light",
        "name": "Hue lightstrip plus 1",
        "modelid": "LST002",
        "manufacturername": "Signify Netherlands B.V.",
        "productname": "Hue lightstrip plus",
        "capabilities": {
            "certified": true,
            "control": {
                "mindimlevel": 40,
                "maxlumen": 1600,
                "colorgamuttype": "C",
                "colorgamut": [
                    [
                        0.6915,
                        0.3083
                    ],
                    [
                        0.17,
                        0.7
                    ],
                    [
                        0.1532,
                        0.0475
                    ]
                ],
                "ct": {
                    "min": 153,
                    "max": 500
                }
            },
            "streaming": {
                "renderer": true,
                "proxy": true
            }
        },
        "config": {
            "archetype": "huelightstrip",
            "function": "mixed",
            "direction": "omnidirectional",
            "startup": {
                "mode": "safety",
                "configured": true
            }
        },
        "uniqueid": "00:17:88:01:04:06:ae:3d-0b",
        "swversion": "1.50.2_r30933",
        "swconfigid": "59F2C3A3",
        "productid": "Philips-LST002-1-LedStripsv3"
    },
    "5": {
        "state": {
            "on": false,
            "bri": 144,
            "hue": 7676,
            "sat": 199,
            "effect": "none",
            "xy": [
                0.5016,
                0.4151
            ],
            "ct": 443,
            "alert": "select",
            "colormode": "xy",
            "mode": "homeautomation",
            "reachable": true
        },
        "swupdate": {
            "state": "noupdates",
            "lastinstall": "2021-08-13T13:53:48"
        },
        "type": "Extended color light",
        "name": "Upstairs Hall",
        "modelid": "LCT015",
        "manufacturername": "Signify Netherlands B.V.",
        "productname": "Hue color lamp",
        "capabilities": {
            "certified": true,
            "control": {
                "mindimlevel": 1000,
                "maxlumen": 806,
                "colorgamuttype": "C",
                "colorgamut": [
                    [
                        0.6915,
                        0.3083
                    ],
                    [
                        0.17,
                        0.7
                    ],
                    [
                        0.1532,
                        0.0475
                    ]
                ],
                "ct": {
                    "min": 153,
                    "max": 500
                }
            },
            "streaming": {
                "renderer": true,
                "proxy": true
            }
        },
        "config": {
            "archetype": "sultanbulb",
            "function": "mixed",
            "direction": "omnidirectional",
            "startup": {
                "mode": "safety",
                "configured": true
            }
        },
        "uniqueid": "00:17:88:01:04:ff:49:53-0b",
        "swversion": "1.88.1",
        "swconfigid": "76B74E79",
        "productid": "Philips-LCT015-1-A19ECLv5"
    },

 

 


I am wanting information for "4" and "5" to be ingested as separate events at index time. I understand that one could use regex to filter this properly, but honestly I'm struggling to wrap my head around how. 

Any help would be massively appreciated.

Many Thanks,

John

Labels (1)
0 Karma
1 Solution

dumdees
Explorer

Absolutely amazing. Can't thank you enough!

I really need to read up more on how Splunk uses regex to split event data like this. 

View solution in original post

0 Karma

somesoni2
Revered Legend

Give this a try

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?=\s+\"\d+\":)
NO_BINARY_CHECK=true
CHARSET=UTF-8
disabled=false
SEDCMD-removeHeader=s/^\{$//
SEDCMD-addHeader=s/^\s+\"/{"/
SEDCMD-addFooter=s/\},$/}\n}/

You'll have to add timestamp parsing config as well (use timestamp on the event OR use current time)

dumdees
Explorer

That's excellent thank you! Got that working, however all the field names have the number in front of them... Any idea how I'd get around this?

 

Screenshot 2022-06-01 at 21.53.42.png

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@dumdees - I've not tested it but try this LINE_BREAKER in the staging/testing environment.

(,*{*[\r\n]+\s+\"\d+\":\s*)

 

I hope this helps!!!

dumdees
Explorer

Absolutely amazing. Can't thank you enough!

I really need to read up more on how Splunk uses regex to split event data like this. 

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...