Getting Data In

How to search for devices/servers not reporting to Splunk?

waJesu
Path Finder

I have two queries I want to merge and I need expert help. The first one returns reporting devices as good and non-reporting devices as missing. The second one returns the missing devices with a heartbeat but not sending logs. Help me come up with one query that would show results for Good, Heartbeat and Missing:

| tstats latest(_time) as latest where index="*" earliest=-5d by host
| eval recent = if(latest > relative_time(now(),"-15m"),"Good","Missing"), realLatest = strftime(latest,"%c")

| tstats latest(_time) as latest where index="_*" earliest=-5d by host
| eval recent = if(latest > relative_time(now(),"-15m"),"Heartbeat","Missing"), realLatest = strftime(latest,"%c")
Labels (1)
Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

See if this helps.

 

 

| tstats latest(_time) as latest where (index="*" OR index=_internal) earliest=-5d by host
| eval recent = case(index!=_internal AND latest > relative_time(now(),"-15m"),"Good", 
                     index!=_internal AND latest <= relative_time(now(),"-15m"), "Missing", 
                     index=_internal AND latest > relative_time(now(),"-15m"), "Heartbeat", 
                     1==1, "Missing"), 
       realLatest = strftime(latest,"%c")

 

 

Standard disclaimer: don't use index=* in production.

---
If this reply helps you, Karma would be appreciated.
0 Karma

waJesu
Path Finder

This is the error I am getting after running the query:

waJesu_0-1685552569002.png

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I dropped a comma in my reply.  It's fixed.

---
If this reply helps you, Karma would be appreciated.
0 Karma

waJesu
Path Finder

Thank you. I think you forgot to attach the corrected query.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Didn't forget.  The correction is in the original query.

---
If this reply helps you, Karma would be appreciated.
0 Karma

waJesu
Path Finder

I am not sure why it's returning "Missing only even on devices that are reporting. Maybe the query needs a tweak?

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Query has been tweaked.

---
If this reply helps you, Karma would be appreciated.
0 Karma

waJesu
Path Finder

Maybe I am missing something. It's still returning "Missing" for everything.

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please have a look at the case function and verify the logic there meets your business requirements.

---
If this reply helps you, Karma would be appreciated.
0 Karma

waJesu
Path Finder

I think it does. We want the query to return devices sending logs as Good, those not reporting as Missing and those missing yet have a heartbeat as Heartbeat. That's what the case function is saying. I am actually surprised I am not getting expected results.

 

0 Karma

waJesu
Path Finder

Good morning.  Any new thoughts as to why my results are showing "Missing" only even for devices/servers I know to be reporting? Anything to tweak the query somehow?

0 Karma

waJesu
Path Finder

Good morning.  Any new thoughts as to why my results are showing "Missing" only even for devices/servers I know to be reporting? Anything to tweak the query somehow?

0 Karma
Get Updates on the Splunk Community!

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

What's New in Splunk Observability Cloud and Splunk AppDynamics - May 2025

This month, we’re delivering several new innovations in Splunk Observability Cloud and Splunk AppDynamics ...