Splunk Search

I need an alert that triggers when it can't find a certain message in the logs within the last hour

New Member

It always brings up no results. Here is my query:

index=abc  host = "123" OR host = "456" OR host = "789" OR host = "012" fullload = "]I:  Task is running" | stats count by source, 
fullload | where count < 1 

It is looking for this message: "]I: Task is running"
can you please help

Tags (3)
0 Karma

SplunkTrust
SplunkTrust

hey you can try something like this
when there is "NO result found" processCount field will get the value of 0

 index=abc  host = "123" OR host = "456" OR host = "789" OR host = "012" fullload = "]I:  Task is running" | stats count by source, fullload   |  appendpipe [ stats count | eval processCount=0 | where count==0 | fields - count ] | search processCount=0

Run this for last 1 hour and schedule an alert to run every hour for last 1 hour .
Trigger alert when select custom and write condition processCount=0

let me know if this helps!

0 Karma

New Member

Thanks I will try it.

0 Karma

New Member

I need the search to bring up all occurrences of a count of 0 messages by source. There are 250 sources.

0 Karma

Champion

If you're going to search for "what's not there", you have to include in your search some default values of what should be there, so that you aren't just missing rows for the missing data. This run anywhere search shows how you can do this:

index=_internal sourcetype=mongod OR sourcetype=splunkd OR sourcetype=splunkd_conf OR sourcetype=madeup
| stats count BY sourcetype
| append [| makeresults | eval sourcetype="mongod,splunkd,splunkd_conf,madeup" | makemv delim="," sourcetype | table sourcetype | mvexpand sourcetype | eval count=0]
| stats sum(count) AS count BY sourcetype
| search count=0

The append line could use inputlookup to fetch the list of expected sourcetypes, or you could hardcode them into the search as shown here. It creates a row for each expected sourcetype with count=0, then performs another stats afterwards to sum them so you can see which events created via append didn't have any additional count from before.

0 Karma

New Member

Thanks I will try it.

0 Karma