Alert if any forwarder service went down

[shivanandbm]

I need to send alerts to a team for below condition

1)if any forwarder went down
2) Forwarder is online But not able to send data.

we have around 1000 forwarders in our environment .If any one could guide me how to set up the alerts for above condition it will be help to us.

[DavidHourani · Accepted solution]

Hi @shivanandbm,

You have two ways to go about this. The easy way would be to leverage the forwarder section of the monitoring console as this information is already available there :
https://docs.splunk.com/Documentation/Splunk/7.3.0/DMC/ForwardersDeployment

The other way would be to run a search such as the one below and then filter only on your forwarders :

 | metadata type=hosts | eval secs_since_last_saw=now()-lastTime

This will give you the time since the last event was seen per forwarder. You can then set a threshold and apply it to find whichever forwarder hasn't been sending for some time.

Cheers,
David