Splunk Search

Data frequency for all index and sourcetype.

arjit
Path Finder

Hi All,  I was looking for a query which can look for the previous one month data, calculate frequency of data being coming to indexer and sourcetype (data is not real time and for eg. say we are getting data into index and sourcetype after . 5 hours or so) and return average (as frequency will differ every time say sometime 5 hours or sometimes 4 hours). For example. Indexer A and sourcetype B are receiving data every 5 hour on an average within a day. Can somebody please help in this regards? Thanks.

Labels (6)
0 Karma
1 Solution

tscroggins
Influencer

The monitoring console provides examples for working with license usage data (Indexing > License Usage > Historic License Usage). You can adapt those searches to your needs.

The average interarrival time is just the inverse of the arrival rate, so we can use e.g. the sum of the bytes indexed every 24 hours to find the average time in hours between bytes. Adapted from the monitoring console:

index=_internal source=*license_usage.log* type=Usage earliest=-1mon@mon latest=@mon
| eval h=if(((len(h) == 0) OR isnull(h)),"(SQUASHED)",h) 
| eval s=if(((len(s) == 0) OR isnull(s)),"(SQUASHED)",s) 
| eval idx=if(((len(idx) == 0) OR isnull(idx)),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time st idx
| stats avg(eval(1/(b/24))) as avg_interarrival_time_in_hours by _time idx st

 

View solution in original post

arjit
Path Finder

Thanks @tscroggins ! I will look into them...

0 Karma

tscroggins
Influencer

The monitoring console provides examples for working with license usage data (Indexing > License Usage > Historic License Usage). You can adapt those searches to your needs.

The average interarrival time is just the inverse of the arrival rate, so we can use e.g. the sum of the bytes indexed every 24 hours to find the average time in hours between bytes. Adapted from the monitoring console:

index=_internal source=*license_usage.log* type=Usage earliest=-1mon@mon latest=@mon
| eval h=if(((len(h) == 0) OR isnull(h)),"(SQUASHED)",h) 
| eval s=if(((len(s) == 0) OR isnull(s)),"(SQUASHED)",s) 
| eval idx=if(((len(idx) == 0) OR isnull(idx)),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time st idx
| stats avg(eval(1/(b/24))) as avg_interarrival_time_in_hours by _time idx st

 

Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...