Data frequency for all index and sourcetype.

[arjit]

Hi All,  I was looking for a query which can look for the previous one month data, calculate frequency of data being coming to indexer and sourcetype (data is not real time and for eg. say we are getting data into index and sourcetype after . 5 hours or so) and return average (as frequency will differ every time say sometime 5 hours or sometimes 4 hours). For example. Indexer A and sourcetype B are receiving data every 5 hour on an average within a day. Can somebody please help in this regards? Thanks.

[arjit]

Thanks @tscroggins ! I will look into them...

[tscroggins · Accepted solution]

The monitoring console provides examples for working with license usage data (Indexing > License Usage > Historic License Usage). You can adapt those searches to your needs.

The average interarrival time is just the inverse of the arrival rate, so we can use e.g. the sum of the bytes indexed every 24 hours to find the average time in hours between bytes. Adapted from the monitoring console:

index=_internal source=*license_usage.log* type=Usage earliest=-1mon@mon latest=@mon
| eval h=if(((len(h) == 0) OR isnull(h)),"(SQUASHED)",h) 
| eval s=if(((len(s) == 0) OR isnull(s)),"(SQUASHED)",s) 
| eval idx=if(((len(idx) == 0) OR isnull(idx)),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time st idx
| stats avg(eval(1/(b/24))) as avg_interarrival_time_in_hours by _time idx st