Splunk Search

How to collect events that form part of a common tree

bowesmana
SplunkTrust
SplunkTrust

I have a large NodeRED JSON flows.json file that I'm ingesting into Splunk. In that file there are one or more 'flows', which are made up of a sequence of 'nodes'.

Each NodeRED 'node' is a JSON snippet and I have configured Splunk to ingest these as separate events. In basic form, they look like this

 

{
    "id": "2e88d163.b8d20e",
    "type": "evaluator",
    "z": "430b6531.d34c7c",
    "name": "",
    "x": 870,
    "y": 300,
    "wires": [
        [
            "c53c6260.e6a06"
        ]
    ]
}

 

where x/y/z are UI related attributes, but ID, type and wires are key to the flow sequence. A node can be connected to any number of other nodes via the 'wires', where the id references the id of another node.

As a bit of an exercise I started to wonder if it was possible to 'transaction' all the nodes involved in a single flow so that all the node objects could then be visualised either in a simple table or a sequence diagram. 

The challenge seems to be that there is no common attribute to join all the nodes together. There can be any number of wires in the array, indicating different paths in the flow and the flow can have as many nodes as it likes.

In my case, it always starts with a particular 'type' and ends with another 'type', so I know when the flow starts and ends.

I did think of putting all this data to a lookup, but I still am not sure if it's possible to collect all nodes in a flow as it seems as though I would need to have an unknown number of passes through the data to fill in the wire connections.

Can anyone think how this could be done?

 

Labels (5)
0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...