We log job status messages in splunk. When a job runs successfully, a success message is logged. When a job errors out, an error message is logged. Both types of messages include hostname as a field. But when the underlying service fails to run a job, no message is logged.
I need to find hostnames that are missing success messages. If I could use dataset literals, I might search something like this:
| FROM <list of expected hostnames as dataset literal> NOT [subsearch for success message hostnames]
But Splunk Cloud Platform apparently does not support the use of dataset literals, so I've resorted to a more convoluted process using stats, as suggested by several Internet authors:
<search for success message hostnames> | eval expected = split("<list of expected hostnames>"," ") | stats values(hostname) as hostname by expected | where NOT match (hostname,expected)
This approach works if some, but not all, expected hostnames are missing. However, in the case where all the expected hostnames are missing the search comes back empty. I understand why it comes back empty. What I need is a "correct" way to find these missing hostnames that will work in all cases.
