topic Re: Alert for Linux server in Alerting

Alert for Linux server

Vishal2 — Fri, 23 Jul 2021 09:54:57 GMT

Can you provide the An example of search query or script. If Linux server is shutdown or down or up. I am looking for the best way to setup an shutdown or down or up status alert for Linux server.

Re: Alert for Linux server

gcusello — Fri, 23 Jul 2021 10:14:54 GMT

Hi @Vishal2,

my hint is to monitor the Splunk Universal Forwarder that takes logs from that server so, when you don't receive Splunk internal logs from that server, you have to intervene because the server is down or there's a problem one the Forwarder or the network that requires yout intervene.

You can monitor a list of hosts (not depending on the operative system) creating a lookup (called e.g. perimeter.csv) contaning only one column (called host) and run a search like the following:

Then you can schedule this search as an alert to run e.g. every five minutes.

Youcan also save this search (without the last row) to display a situation of your infrastructure (total=0 means down, total>0 means up).

Ciao.

Giuseppe

Re: Alert for Linux server

Vishal2 — Mon, 26 Jul 2021 18:06:52 GMT

Hi,

I know that, you are talking about the logs monitoring but if universal forwarer is failed that time logs not coming to splunk, I don't need that, I need server shutdown or down related query

Is it possible to write the query for Linux server down or shutdown or up ??? If it is possible what is that query ??

Re: Alert for Linux server

gcusello — Tue, 27 Jul 2021 12:34:36 GMT

Hi @Vishal2,

the message to search when a server is shutting down is "The system is going down", so you can run something like this:

index=os "The system is going down"

if you have results there was a shutdown, but this doesn't say if the server was still down or not.

In addition , if you don't have the message, it's possible that there was a crash or the Forwarder is down.

For this reason I hint to check the Splunk internal logs.

Ciao.

Giuseppe