How to detect a host which stops sending logs?

[bbiandov]

Hi everyone, How do I detect a host which stops sending logs? Ideally an alert would be the optimal application of an elaborate search. So far |metadata type=hosts | eval age = now() - lastTime | search age > 86400 | sort age d | convert ctime(lastTime) | fields age,host,lastTime doesn't do the trick.

It only works for one day and then it stops producing results (we are assuming the host which stopped forwarding stays that way meaning it worked once upon a time and then it stopped working; it doesn't just start working again by itself)...hence the need to detect this condition MORE THAN a day after the host stopped forwarding. For example I want the search to produce result if a host last forwarded a log 28 days ago and since then it hasn't forwarded anything.

Thank you!

[jd]

You can use an app named : "deployement monitor"

[linu1988]

you need to run over 2/3 days to get the missing forwarder alert again and again. The problem with daily alert is metdata doesn't show you older records.

[bbiandov]

Thanks gkanapathy, the challenge I am facing is that the search works just fine; but if saved as an alert one can ONLY select 1 day; sure you can change it in the GUI but once saved it reverts back to 1 day. That's what breaks it:

https://www.dropbox.com/s/2dzon27ssowqicg/splunk.png?dl=0

[gkanapathy]

you need to edit the search, not the time selector.

[gkanapathy]

Seems like you should just be able to increase the number you're comparing against age, e.g., to 86400 x 28. I suppose if you want to every day report on every host that has ever stopped sending, then you can just remove that filter entirely.