Getting Data In

Structuring nested data

Tim
Explorer

Hi all,

I have an application that needs to write some data that may be several levels deep and I'm struggling to understand how I can best represent it for Splunk.

Here's an example of a data set. It consists of:

DeviceID=nnnnn

Application=App1

    Permissions=X,Y,Z

Application=App2

    Permissions=Y,Z

In other words, a device may have multiple applications and each app may have multiple permissions. If I wanted to write a report that list all applications that contain Permission=X, how do you recommend we model the data?

Any guidance you can provide will be greatly appreciated!

Tags (1)
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Easiest by far is to denormalize each fact (i.e., permission or set of permissions) into a self-contained and fully-qualifed distinct record, e.g. (each line below is a separate event/record):

DeviceID=nnnn,Application=App1,Permission=X
DeviceID=nnnn,Application=App1,Permission=Y
DeviceID=nnnn,Application=App1,Permission=Z
DeviceID=nnnn,Application=App2,Permission=Y
DeviceID=nnnn,Application=App2,Permission=Z

The Splunk query for applications containing permission X would be Permission=X | stats count by Application (which would give you the count for each application as well, which you could discard.)

You could compact it up one level, though for some reporting purposes you may wind up expanding that one level out to the equivalent of the above at search time (in the specific case of this report, though Permission=X | stats count by Application will still work the same):

DeviceID=nnnn,Application=App1,Permission=X,Permission=Y,Permission=Z
DeviceID=nnnn,Application=App2,Permission=Y,Permission=Z

Note, btw, that while the raw forms suggested here look much larger than a tree, the indexed and compressed data set (which is how it would be stored on disk by Splunk) for large sets of data is not larger than if it were represented as a tree or a more normalized form.

smolcj
Builder

There isn't any concept of nested table in splunk?

0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...