topic Difference between NOT and isnotnull in Splunk Search

Difference between NOT and isnotnull

rockosmodernlif — Tue, 16 Jul 2019 19:23:42 GMT

First of all, I'm a noob with Splunk and I started doing the fundamentals training.

I'm at the logical operators module, and the following question arise from there.

I have http request events that I want to filter out based on whether or not a request header key exists, in my case request.headers.bot. The value of this field does not matter. This field only exists for requests flagged as bots. Regular requests will not have this field.

event:

{
  id: 123
  request: {
    headers: {
      cookie: "key=value; something=else;"
      user-agent: "Mozilla/5.0"
      bot: "yes"
    }
    path: "/"
  }
  time: 2019-07-16T18:08:59.980Z
}

so, I'm running a search query to find out how many events are not bots:

before the training I had something like this:

request.path="/" | where isnull('request.headers.bot')

but, while doing the training, I found out you could also do:

request.path="/" NOT request.headers.bot="*"

according to the training video, NOT returns events where the field does not exist or does not have the value specified

However, for the same period of time, I'm getting different results:
total events: 4000
isnull: 3778
not: 3798

shouldn't both queries return the same events?

also, is there a query that I can use to find the missing 20 events between the first and second query?

thanks!

Re: Difference between NOT and isnotnull

nick405060 — Tue, 16 Jul 2019 23:38:02 GMT

No, they should not produce the same events. A bit of background, != excludes null events (e.g. myfield!="asdf" is going to also discard null events), where NOT does not do this, it keeps the null events (e.g. NOT myfield="asdf"). It's poorly designed in my opinion and very dangerous; I had live dashboards for OVER A YEAR that were misrepresenting data because I was using != and did not want events with null values for a specific field discarded.

So, more relevant, there is a difference between where and search. search myfield="*" is going to search for everything, whereas where myfield="*" is going to look for the literal *

So assuming this is in a where clause, your second statement is just going to display all your results (assuming request.headers.bot is never a literal ) whereas the first statement is going to correctly only display null. You should be able to do either ```where NOT request.headers.bot=,where isnull(request.headers.bot),search isnull(request.headers.bot),search request.headers.bot!="", orsearch NOT request.headers.bot=""``` (I believe)

Re: Difference between NOT and isnotnull

nick405060 — Tue, 16 Jul 2019 23:46:18 GMT

One more thing - null is different from an empty string

Re: Difference between NOT and isnotnull

nick405060 — Tue, 16 Jul 2019 23:53:48 GMT

tl;dr remove your quotes around the *

sorry for the essay

Re: Difference between NOT and isnotnull

MuS — Wed, 17 Jul 2019 01:00:05 GMT

Just as addition to this answer here is the docs link https://docs.splunk.com/Documentation/Splunk/latest/Search/NOTexpressions with some good examples.

And regarding the search vs where use search if you want to search (HaHA) for a field value and use where if you want to compare two fields or use eval() functions on a field, see the docs https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/search#Comparing_two_fields

cheers, MuS