Splunk Search

Data frequency for all index and sourcetype.

arjit
Path Finder

Hi All,  I was looking for a query which can look for the previous one month data, calculate frequency of data being coming to indexer and sourcetype (data is not real time and for eg. say we are getting data into index and sourcetype after . 5 hours or so) and return average (as frequency will differ every time say sometime 5 hours or sometimes 4 hours). For example. Indexer A and sourcetype B are receiving data every 5 hour on an average within a day. Can somebody please help in this regards? Thanks.

Labels (6)
0 Karma
1 Solution

tscroggins
Motivator

The monitoring console provides examples for working with license usage data (Indexing > License Usage > Historic License Usage). You can adapt those searches to your needs.

The average interarrival time is just the inverse of the arrival rate, so we can use e.g. the sum of the bytes indexed every 24 hours to find the average time in hours between bytes. Adapted from the monitoring console:

index=_internal source=*license_usage.log* type=Usage earliest=-1mon@mon latest=@mon
| eval h=if(((len(h) == 0) OR isnull(h)),"(SQUASHED)",h) 
| eval s=if(((len(s) == 0) OR isnull(s)),"(SQUASHED)",s) 
| eval idx=if(((len(idx) == 0) OR isnull(idx)),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time st idx
| stats avg(eval(1/(b/24))) as avg_interarrival_time_in_hours by _time idx st

 

View solution in original post

arjit
Path Finder

Thanks @tscroggins ! I will look into them...

0 Karma

tscroggins
Motivator

The monitoring console provides examples for working with license usage data (Indexing > License Usage > Historic License Usage). You can adapt those searches to your needs.

The average interarrival time is just the inverse of the arrival rate, so we can use e.g. the sum of the bytes indexed every 24 hours to find the average time in hours between bytes. Adapted from the monitoring console:

index=_internal source=*license_usage.log* type=Usage earliest=-1mon@mon latest=@mon
| eval h=if(((len(h) == 0) OR isnull(h)),"(SQUASHED)",h) 
| eval s=if(((len(s) == 0) OR isnull(s)),"(SQUASHED)",s) 
| eval idx=if(((len(idx) == 0) OR isnull(idx)),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time st idx
| stats avg(eval(1/(b/24))) as avg_interarrival_time_in_hours by _time idx st

 

View solution in original post

Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!