Logging frequency of my index,sourcetype and host

[thippeshaj]

Hi Splunkers,

How do I calculate the logging frequency of my index=xxx sourcetype=yyy host=zzz?

Explanation: I have a different set of logs which sends logs with different frequency, some of them send every minute/hour and some of them send the logs once a day. so basically logging frequency is not fixed, it's dynamic.

I'm trying to find out a way to alert if a particular index=xxx sourcetype=yyy host=zzz stops sending logs, I want a dynamic way of calculating the frequency threshold wherein I can say (now()-last_event_time) > threshold, I don't want to use something like which is basically find the difference between the last event time VS the current and some random threshold.

I want Splunk to tell the ideal threshold for my index,sourcetype and host combination.
For example; A particular logs from index=a host=b sourceype=c logs once in a day, so here I want Splunk to tell an ideal threshold I can use ( maybe ~ 1day or ~ 24 hours) as the threshold to set an alert.
another example, a particular logs from index=g sourcetype=h host=i logs every 4.5/5 hours, so here I want Splunk to tell an ideal threshold I can use ( maybe ~ 4H or ~ 4.5h or 5h or xh) as the threshold to set an alert.

so using this I can set an alert like (now()-last_event_time) > threshold

Thanks in advance.

Happy Splunking.

[amehlmann]

Hi,

you can calculate the average timespan between events using

| tstats count as event_count, latest(_time) as latest_time, earliest(_time) as earliest_time by host,index
| eval total_time_spans = latest_time - earliest_time
| eval average_time_span = total_time_spans / (event_count - 1)
| stats avg(average_time_span) as avg_time_span by host,index

But beware since this makes only sense if you have regular reporting hosts/indexes. this will not work if one host e.g. sends 1k events once a day.

[MKozanic]

Hi @thippeshaj ,

Just wondering if you worked out a solution for this?

I'm currently looking for something similar to be able to configure alerts based on previous indexing frequency

[dmuralidaran_sp]

How about configuring an alert to see if you get new data every interval according to the configuration for each host machine?