Splunk Search

How can I get the occurrence of a field in events as a percentage, when the field names are unknown (dynamic per event)?

nicholmikey
Explorer

I have events with JSON in them and I need to know what % of the time each field appears.

The fieldset in the events is not consistent, sometimes an event has many, sometimes only a few, the name of each field is unknown at the time of the search

So far I have used rex to extract the JSON, and spath to extract the fields from the JSON. I also used fields - so now the events only have the fields I am interested in. Other than the Time field, if I remove that I get no results.

How can I generate a table that shows
Field a appears: 40%
Field b appears: 80%
Field c appears 10%
So on...

The fields are dynamic in name and occurrence, so I don't know the names at the time of the search.

Is there some way to accomplish this?

Thanks,

0 Karma
1 Solution

to4kawa
Ultra Champion
...
|fieldsummary

View solution in original post

to4kawa
Ultra Champion
...
|fieldsummary

nicholmikey
Explorer

Hi @to4kawa thanks for that it gets me most of the way there. But I see it does not show a % of how many times that field appears in the events, it only shows a count. Is there a way to add a column with the total count of events, including events where that field did not appear, so that I can do an eval and get a percentage?

0 Karma

to4kawa
Ultra Champion
| fieldsummary
| table field count
| eventstats max(count) as total
| eval perc= count / total * 100

nicholmikey
Explorer

Fantastic. I learned a lot. Thanks.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...