Splunk Search

Average number of fields in an event

ewanbrown
Path Finder

We use splunk to index beacons our application sends in, many of these fields are optional, and we'd like to calculate the average number of fields that we are sent, I've not been able to work this out, and have this so far. (and have tried variants of it)

index=Beacon | fieldsummary | stats values(field), count(field), avg(count(field))

Thanks

Tags (2)
0 Karma
1 Solution

jplumsdaine22
Influencer

The highest value of the field count in the results for fieldsummary should be the same as the events returned by the query (as every event has an index field) As such the following query should give you the mean number of fields:

index=Beacon | fieldsummary |  stats sum(count) as total_fields max(count) as total_events | eval avg_fields=total_fields/total_events

you could modify the fields being counted by placing | search NOT foo=* between fieldsummary and stats to exclude fields such as timestamp fields that are common to all events. Just make sure you leave at least one metadata field so stats can figure out the total number of events.

View solution in original post

jplumsdaine22
Influencer

The highest value of the field count in the results for fieldsummary should be the same as the events returned by the query (as every event has an index field) As such the following query should give you the mean number of fields:

index=Beacon | fieldsummary |  stats sum(count) as total_fields max(count) as total_events | eval avg_fields=total_fields/total_events

you could modify the fields being counted by placing | search NOT foo=* between fieldsummary and stats to exclude fields such as timestamp fields that are common to all events. Just make sure you leave at least one metadata field so stats can figure out the total number of events.

ewanbrown
Path Finder

Thanks this worked

0 Karma

renjith_nair
Legend

Does this work for you?

index=Beacon| eval filedlist=split(_raw," ") | eval fieldcount=mvcount(filedlist)|table filedlist fieldcount|stats avg(fieldcount)
---
What goes around comes around. If it helps, hit it with Karma :slightly_smiling_face:
0 Karma

ewanbrown
Path Finder

Thanks, it was almost right, but we have a URL we extract into different fields and it counted that as one, but we wanted each extracted part to be counted

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...