Is there a dashboard to monitor when event data is no longer being sent to Splunk from our hosts?

[bluemarvel]

Hello,

I am looking for a specific dashboard to detect when an event/log source is no longer sending events. The dashboard has to show and/or indicate with host is no longer sending events. I have something like this as an example, it of course does not work.

index=firewall host=servername | eval status=case (Count < 3,"Host Stopped Sending Data") | table host status Count  |  dedup  1 host  | sort  by status, count

[somesoni2]

Give this a try (assuming threshold is time, like no data came in last 30 mins)

| metadata type=hosts index=firewall | where host="yourhostname" | eval status=case (recentTime<relative_time(now(),"-30m"),"Host Stopped Sending Data","All is well") | table host status totalCount

[becksyboy]

Hi @somesoni2 when i try this, i get "Error in 'eval' command: The arguments to the 'case' function are invalid." do you know why this is the case?

[gjanders]

This post is from 2016...Here's some info from community slack..

There are a lot of options for finding hosts or sources that stop submitting events:
Meta Woot! https://splunkbase.splunk.com/app/2949/
TrackMe https://splunkbase.splunk.com/app/4621/
Broken Hosts App for Splunk https://splunkbase.splunk.com/app/3247/
Alerts for Splunk Admins ("ForwarderLevel" alerts) https://splunkbase.splunk.com/app/3796/
Monitoring Console https://docs.splunk.com/Documentation/Splunk/latest/DMC/Configureforwardermonitoring
Deployment Server https://docs.splunk.com/Documentation/DepMon/latest/DeployDepMon/Troubleshootyourdeployment#Forwarde...

Some helpful posts:
https://lantern.splunk.com/hc/en-us/articles/360048503294-Hosts-logging-data-in-a-certain-timeframe
https://www.duanewaddle.com/proving-a-negative/

[becksyboy]

Thanks, some useful links here.

[sundareshr · Accepted solution]

If the firewall index has data from only one source, you could try this.

index=firewall | stats count by host | where count<3

[ddrillic]

The improved DMC has the view of forwarders which haven't phoned home in the past hour/day, etc. Would it be good for you?