Getting Data In

Search for results NOT found in the last 24hours?

ericsales
New Member

Edit: rephrasing the question a bit

I have a job that is remotely triggered which should be run at least once within a 24 hour period. The start message (i.e. "Job Triggered") appears in /var/log/messages. What is the optimal way to search/report for hosts that DO NOT have the Job Triggered message within a 24 hour period?

So far, I have this in the search cmd:
source="/var/log/messages" host="*" "Job Triggered." earliest=-1d | dedup host | stats count by host

This shows the results, but doesn't tell me how many hosts didn't have the Job Triggered in that period.

Tags (1)
0 Karma

sowings
Splunk Employee
Splunk Employee

In order to evaluate against history (to find a gap), you'll have to collect some history. A way that this is achieved in the Deployment Monitor app (which ships with Splunk) is to utilize a summary index that's used to "remember what is seen". Another way would be to use | inputlookup combined with | outputlookup to create a CSV file that has some history.

Ultimately, you'd end up with a list of "hosts we've seen kick off the job over all time, and the last time they ran it", and then perform some time math against | eval this_time=now() to see if it's > 24h.

Get Updates on the Splunk Community!

Splunk Lantern | Spotlight on Security: Adoption Motions, War Stories, and More

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Cloud | Empowering Splunk Administrators with Admin Config Service (ACS)

Greetings, Splunk Cloud Admins and Splunk enthusiasts! The Admin Configuration Service (ACS) team is excited ...

Tech Talk | One Log to Rule Them All

One log to rule them all: how you can centralize your troubleshooting with Splunk logs We know how important ...