topic Re: Monitoring log sources that go silent? in Monitoring Splunk

Monitoring log sources that go silent?

dionrivera — Fri, 04 Nov 2022 19:10:45 GMT

Hi team. I'm looking for a query/solution that will alert me when a log source is no longer sending logs. For example, I have an index called "linux_prod" which is populated when linux hosts fortheir events. I would like to receive an alert when this index stops receiving events for the past 1 hour. This happens when SC4S or some other issue on the network have problems.

Thank you.

Re: Monitoring log sources that go silent

gcusello — Fri, 04 Nov 2022 19:10:57 GMT

Hi @dionrivera,

you have to list the indexes to monitor and put them in a lookup, called e.g. perimeter.csv containing at least one column (called e.g. index).

then you could run something like this:

Ciao.

Giuseppe

Re: Monitoring log sources that go silent

dionrivera — Fri, 04 Nov 2022 20:00:15 GMT

@gcuselloAny suggestions how I could include the time within the query? I need to look every hour if the event count has changed.

Grazie!

Re: Monitoring log sources that go silent

gcusello — Sat, 05 Nov 2022 10:17:33 GMT

Hi @dionrivera,

you can run the search using as time the latest hour or insert in the main search "earliest=-h latest=now"

and schedule your alert every hour with cron:

0 * * * *

Ciao.

Giuseppe

Re: Monitoring log sources that go silent?

dionrivera — Sun, 06 Nov 2022 22:57:55 GMT

Thank you sir. Much appreciated.

Re: Monitoring log sources that go silent?

gcusello — Mon, 07 Nov 2022 07:17:16 GMT

Hi @dionrivera,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉