topic Data frequency for all index and sourcetype. in Splunk Search

Data frequency for all index and sourcetype.

arjit — Fri, 26 Mar 2021 23:42:42 GMT

Hi All, I was looking for a query which can look for the previous one month data, calculate frequency of data being coming to indexer and sourcetype (data is not real time and for eg. say we are getting data into index and sourcetype after . 5 hours or so) and return average (as frequency will differ every time say sometime 5 hours or sometimes 4 hours). For example. Indexer A and sourcetype B are receiving data every 5 hour on an average within a day. Can somebody please help in this regards? Thanks.

Re: Data frequency for all index and sourcetype.

tscroggins — Sun, 28 Mar 2021 00:32:27 GMT

The monitoring console provides examples for working with license usage data (Indexing > License Usage > Historic License Usage). You can adapt those searches to your needs.

The average interarrival time is just the inverse of the arrival rate, so we can use e.g. the sum of the bytes indexed every 24 hours to find the average time in hours between bytes. Adapted from the monitoring console:

index=_internal source=*license_usage.log* type=Usage earliest=-1mon@mon latest=@mon | eval h=if(((len(h) == 0) OR isnull(h)),"(SQUASHED)",h) | eval s=if(((len(s) == 0) OR isnull(s)),"(SQUASHED)",s) | eval idx=if(((len(idx) == 0) OR isnull(idx)),"(UNKNOWN)",idx) | bin _time span=1d | stats sum(b) as b by _time st idx | stats avg(eval(1/(b/24))) as avg_interarrival_time_in_hours by _time idx st

Re: Data frequency for all index and sourcetype.

arjit — Sun, 28 Mar 2021 00:49:55 GMT

Thanks @tscroggins ! I will look into them...