How can I get the occurrence of a field in events as a percentage, when the field names are unknown (dynamic per event)?

[nicholmikey]

I have events with JSON in them and I need to know what % of the time each field appears.

The fieldset in the events is not consistent, sometimes an event has many, sometimes only a few, the name of each field is unknown at the time of the search

So far I have used rex to extract the JSON, and spath to extract the fields from the JSON. I also used fields - so now the events only have the fields I am interested in. Other than the Time field, if I remove that I get no results.

How can I generate a table that shows
Field a appears: 40%
Field b appears: 80%
Field c appears 10%
So on...

The fields are dynamic in name and occurrence, so I don't know the names at the time of the search.

Is there some way to accomplish this?

Thanks,

[to4kawa · Accepted solution]

...
|fieldsummary

[nicholmikey]

Hi @to4kawa thanks for that it gets me most of the way there. But I see it does not show a % of how many times that field appears in the events, it only shows a count. Is there a way to add a column with the total count of events, including events where that field did not appear, so that I can do an eval and get a percentage?

[to4kawa]

| fieldsummary
| table field count
| eventstats max(count) as total
| eval perc= count / total * 100

[nicholmikey]

Fantastic. I learned a lot. Thanks.