Search and Alert produce different results

[jlbark]

When I type this search in the normal Splunk search app, I get normal expected results:

"usb" | transaction host startswith="New USB device found" endswith="USB disconnect" | search NOT keyboard NOT mouse NOT host=xyz

However when I create an alert with the same search pattern, I constantly get results for the host xyz.

I am using the newest Splunk 5.0.3. Is this a bug or is there something happening behind the scenes I don't understand?

The settings for the alert are thus:
time range: real time
alert mode: once per search
condition: always
alert action: send email

Any hints?

[gordo32]

I've seen this same question come up a couple times, and my solution is different, so thought I'd share on a few of these in case others have the same problem I did.

The problem was that the query in my Alert was "search index=myindex sourcetype=waf httpstatus=400".

As soon as I removed the keyword "search" from the beginning of this query in the alert, it produced results consistent with manually issuing the search (index=myindex sourcetype=waf httpstatus=400). The rationale behind this (if I understood the support engineer correctly) is that the Alert passes the query to the CLI (i.e. /bin/splunk search ), so the CLI interprets the "search" item in my query as a searchable word, not a function.

[jlbark · Accepted solution]

So, through trial an error, I fixed the problem. There was nothing wrong with the search syntax.

I used the same query, on the search app, and then clicked create -> alert. And created the alert that way. Instead of the previous way, which was to go into the Manager -> Searches and Reports and click "New". For some weird reason I know get the correct alerts. I don't know why this made such a big difference, must be a bug.

[jtrucks]

Perhaps quote the server name and, also, try putting your NOT parameters in the beginning with the "usb" search like this:

NOT keyboard NOT mouse NOT host="xyz" "usb" | transaction host startswith="New USB device found" endswith="USB disconnect"

Unless you need to have the keyboard and mouse negations at the end to make the transaction work. Then do:

NOT host="xyz" "usb" | transaction host startswith="New USB device found" endswith="USB disconnect" | NOT keyboard NOT mouse

Another option is to use xyz* in case it's catching FQDN entries of some sort, which looks like:

"usb" | transaction host startswith="New USB device found" endswith="USB disconnect" | search NOT keyboard NOT mouse NOT host="xyz*"

Anything like these work?

[jlbark]

thanks for your reply jtrucks, but neither of those seemed to do the trick. This is a very weird problem, because the search query works fine in the search app, but the alert seems like it is having the problem.