Splunk Search
Highlighted

Search and Alert produce different results

Explorer

When I type this search in the normal Splunk search app, I get normal expected results:

"usb" | transaction host startswith="New USB device found" endswith="USB disconnect" | search NOT keyboard NOT mouse NOT host=xyz

However when I create an alert with the same search pattern, I constantly get results for the host xyz.

I am using the newest Splunk 5.0.3. Is this a bug or is there something happening behind the scenes I don't understand?

The settings for the alert are thus:
time range: real time
alert mode: once per search
condition: always
alert action: send email

Any hints?

0 Karma
Highlighted

Re: Search and Alert produce different results

Splunk Employee
Splunk Employee

Perhaps quote the server name and, also, try putting your NOT parameters in the beginning with the "usb" search like this:

NOT keyboard NOT mouse NOT host="xyz" "usb" | transaction host startswith="New USB device found" endswith="USB disconnect"

Unless you need to have the keyboard and mouse negations at the end to make the transaction work. Then do:

NOT host="xyz" "usb" | transaction host startswith="New USB device found" endswith="USB disconnect" | NOT keyboard NOT mouse

Another option is to use xyz* in case it's catching FQDN entries of some sort, which looks like:

"usb" | transaction host startswith="New USB device found" endswith="USB disconnect" | search NOT keyboard NOT mouse NOT host="xyz*"

Anything like these work?

0 Karma
Highlighted

Re: Search and Alert produce different results

Explorer

thanks for your reply jtrucks, but neither of those seemed to do the trick. This is a very weird problem, because the search query works fine in the search app, but the alert seems like it is having the problem.

0 Karma
Highlighted

Re: Search and Alert produce different results

Explorer

So, through trial an error, I fixed the problem. There was nothing wrong with the search syntax.

I used the same query, on the search app, and then clicked create -> alert. And created the alert that way. Instead of the previous way, which was to go into the Manager -> Searches and Reports and click "New". For some weird reason I know get the correct alerts. I don't know why this made such a big difference, must be a bug.

View solution in original post

0 Karma
Highlighted

Re: Search and Alert produce different results

Path Finder

I've seen this same question come up a couple times, and my solution is different, so thought I'd share on a few of these in case others have the same problem I did.

The problem was that the query in my Alert was "search index=myindex sourcetype=waf httpstatus=400".

As soon as I removed the keyword "search" from the beginning of this query in the alert, it produced results consistent with manually issuing the search (index=myindex sourcetype=waf httpstatus=400). The rationale behind this (if I understood the support engineer correctly) is that the Alert passes the query to the CLI (i.e. /bin/splunk search ), so the CLI interprets the "search" item in my query as a searchable word, not a function.

0 Karma