How to send an alert when nothing has been written to a specific queue in X minutes?

[cfd0417]

I am looking to set up an alert that will trigger when no messages have been sent to a queue in the last X number of minutes. Does any one have a sample of a similar alert? Thanks in advance!!

[gcusello]

Hi @cfd0417,

when youspeak of a queue, I suppose that you're meaning a Splunk index where logs are stored, is it correct?

if this is your need, you have to create a simple search like the following:

index=your_index

setting the time period as the number of minutes that you need and where your_index is the index where your logs are stored.

Then you have to save this searchas an alert using as trigger condition: results=0.

Then you can configure the actions you need (an eMail, a script execution ,etc...)

Ciao.

Giuseppe

[PickleRick]

You have to remember though that searching through an index and looking only at _time field may yield confusing results sometimes. With badly configured sources/inputs you can have events appearing in the future. So if you have events coming with +1 hour offset and want to find if there have been any events in last 15 minutes, you'll notice that events stopped appearing after 75 minutes. I know that this is a situation which normally should not happen with properly configured infrastricture but I've seen it happen.

The  other case - this time not resulting from misconfiguration but from the way things work - is that some sources might report events with a delay. Sometimes due to buffering or throughput constraints, sometimes they just work in batches and whatnot. But as a result you get a more or less constant stream of events but they are delayed. So everything might be working OK and you might be receiving the events as usual but the events themselves might be, for example, from several hours ago.

So it's always worth checking _indextime as well as _time and consider your typical latency.

[cfd0417]

Thanks Giuseppe. 

[gcusello]

Hi @cfd0417,

if this answer solves your need, please accept it for the other people of Community, otherwise, tell me how  can i help you more on this question.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉