Alerting

Logging frequency of my index,sourcetype and host

thippeshaj
Explorer

Hi Splunkers,

How do I calculate the logging frequency of my index=xxx sourcetype=yyy host=zzz?

Explanation: I have a different set of logs which sends logs with different frequency, some of them send every minute/hour and some of them send the logs once a day. so basically logging frequency is not fixed, it's dynamic.

I'm trying to find out a way to alert if a particular index=xxx sourcetype=yyy host=zzz stops sending logs, I want a dynamic way of calculating the frequency threshold wherein I can say (now()-last_event_time) > threshold, I don't want to use something like which is basically find the difference between the last event time VS the current and some random threshold.

I want Splunk to tell the ideal threshold for my index,sourcetype and host combination.
For example; A particular logs from index=a host=b sourceype=c logs once in a day, so here I want Splunk to tell an ideal threshold I can use ( maybe ~ 1day or ~ 24 hours) as the threshold to set an alert.
another example, a particular logs from index=g sourcetype=h host=i logs every 4.5/5 hours, so here I want Splunk to tell an ideal threshold I can use ( maybe ~ 4H or ~ 4.5h or 5h or xh) as the threshold to set an alert.

so using this I can set an alert like (now()-last_event_time) > threshold

Thanks in advance.

Happy Splunking.

Labels (1)
0 Karma

amehlmann
Engager

Hi,

 

you can calculate the average timespan between events using

| tstats count as event_count, latest(_time) as latest_time, earliest(_time) as earliest_time by host,index
| eval total_time_spans = latest_time - earliest_time
| eval average_time_span = total_time_spans / (event_count - 1)
| stats avg(average_time_span) as avg_time_span by host,index

 

But beware since this makes only sense if you have regular reporting hosts/indexes. this will not work if one host e.g. sends 1k events once a day.

0 Karma

MKozanic
Path Finder

Hi @thippeshaj ,

Just wondering if you worked out a solution for this?

I'm currently looking for something similar to be able to configure alerts based on previous indexing frequency

0 Karma

dmuralidaran_sp
Splunk Employee
Splunk Employee

How about configuring an alert to see if you get new data every interval according to the configuration for each host machine?

0 Karma
Get Updates on the Splunk Community!

ATTENTION!! We’re MOVING (not really)

Hey, all! In an effort to keep this Slack workspace secure and also to make our new members' experience easy, ...

Splunk Admins: Build a Smarter Stack with These Must-See .conf25 Sessions

  Whether you're running a complex Splunk deployment or just getting your bearings as a new admin, .conf25 ...

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...