Don't know why alert is randomly triggered (when no logfile is written in the last hour)

[nessaner]

I have a folder with logs, every hour one logfile is written to it. 
I also have an alert that is triggered when no file is written in the last hour (checking 15 minutes post a hour)
Query:

index=xyz sourcetype=abc
| eval since = now() - _time
| search since < 3600

Mostly it works but sometimes it's triggered even if I can see in the history that logfile for that hour is present in splunk with acurate _time and nothing is missing

What could be the problem?

[gcusello · Accepted solution]

Hi @nessaner,

are you sure that the file is written in the first 15 minutes of each hour?

You can check this issue viewing the _indextime field for the wrong results.

Anyway, you could run your alert after e.g. 30 minutes instead 15:

index=xyz sourcetype=abc earliest=-1h@h latest=@h
| eval since = now() - _time
| search since < 3600

Ciao.

Giuseppe

[nessaner]

Thanks, I cheched intextime and some actually was delayed up to couple of hours! And for most it was only seconds. Not sure why

[gcusello]

Hi @nessaner,

it's usual approving the solution hinted by someone else not your own, but it isn't relevant, the most important thing is that your question has a solution.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉

[gcusello]

Hi @nessaner,

if one answer solves your need, please, accept one answer for the other people of Community or tell me how we can help you.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉