I want to use Splunk to tell me when a process is missing from a list of expected processes.
I have tried using eval and stats, but I can't figure out how to get it right. It is very easy to see when things are there, but hard to tell when they are not there.
This problem can be solved by leveraging a lookup table and a pretty simple search.
First, create a lookup file with your desired processes:
proc,Never Showing Up
proc,Splunk Can Monitor Things
| inputlookup monitor_me.csv
| join type=outer process
[ search sourcetype= proc
| stats count by process]
| table proc count
| fillnull value=0
| where count=0
The search results will look something like this, if the last two entries in the csv are not found:
Never Showing Up 0
Splunk Can Monitor Things 0
Now, you can schedule this search and set up an alert.
View solution in original post