Alerting

How to create an alert if no data is generated from a host?

ashari
Explorer

I want to run a search in splunk to find out that all the devices attached to the splunk server are generating logs. If I dont hear from a device I should receive a alert.

thanks

Tags (1)

martin_mueller
SplunkTrust
SplunkTrust

Here's a quick way to check for hosts not having sent data in a while:

| tstats latest(_time) as latest where index=* by host | where latest < relative_time(now(), "-1d")

Change the -1d according to your needs, run the above search over at least twice the scheduled interval.

koshyk
Super Champion

We had similar siutation and a heartbeat from the other system comes once in every 5 minute. So I'm doing that as a "saved Search" and run every 1 minute and checks data for the previous 6 minutes to see if there is a heartbeat. Else alert.

The logic is:

index=myindex earliest=-6m@s <your_some_more_Search> | stats count as COUNT_HEARTBEAT | where COUNT_HEARTBEAT=0 
0 Karma

MuS
SplunkTrust
SplunkTrust

Hi ashari,

check out the Splunk Deployment Monitor there you can find some nice saved searches related to this topic ... keyword: missing forwarders

hope that helps ...

cheers, MuS

0 Karma

MuS
SplunkTrust
SplunkTrust

sure, although there are a lot of search macros in the saved searches. If you look at the searches you will get an idea how it can be done.
Otherwise use a lookup file which contains all your forwarder hostnames and check if all of them were seen in the past x minutes. Take a look at this http://answers.splunk.com/answers/73268/search-for-hosts-in-a-lookup-but-not-in-splunk

ashari
Explorer

can we do this without splunk deployment monitor app.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...