topic Re: All results are not returned with multiple field exclusions in Splunk Search

All results are not returned with multiple field exclusions

sarahalhawi — Tue, 12 Jul 2016 14:26:32 GMT

Hello,

I am having some issues with using multiple field exclusions as not all results are being returned (only the results for the last 2 days appear).

EVT*-XXXX search eventtype=XXXXX | table txid NOT "vsp-vendor-id=XXXXXXXXXXXXXX"

If I just exclude certain hosts, I get all the required results. However, when I add the vendor id exclusion, only results for the past 2 days appear.

Any ideas why?

Re: All results are not returned with multiple field exclusions

somesoni2 — Tue, 12 Jul 2016 16:51:28 GMT

Try running following and see if you really have any data before past 2 days

EVT*-XXXX [search eventtype=XXXXX | table txid] (host!=XXXX OR host!=XXXX) "vsp-vendor-id=XXXXXXXXXXXXXX"

Re: All results are not returned with multiple field exclusions

sarahalhawi — Wed, 13 Jul 2016 13:28:18 GMT

Yes, this works and returns data for more than 2 days ago. However, as soon as I add another filter, e.g. another vendor id to exclude, only results for 2 days appear. Could this be a limitation of the sub-query or it is something configurable?

Re: All results are not returned with multiple field exclusions

woodcock — Wed, 13 Jul 2016 13:54:54 GMT

It has to be that you have a typo. If you cut and paste both a supposedly matching event and cut and paste your ACTUAL search, it should be easy to tell.

Re: All results are not returned with multiple field exclusions

somesoni2 — Wed, 13 Jul 2016 14:20:05 GMT

The subsearch do have limitation in terms of number of rows and time to finalize. Is there a way to avoid the subsearch? Could you explain the requirement here?

Re: All results are not returned with multiple field exclusions

sarahalhawi — Wed, 13 Jul 2016 14:40:22 GMT

Possibly, what I want to be able to do is filter out a couple of values from different fields (e,g, 2 vendor ids and 2 hosts)

Re: All results are not returned with multiple field exclusions

somesoni2 — Wed, 13 Jul 2016 14:42:47 GMT

What's the purpose of subsearch? OR it's just the formatting issue while posting question.
[search eventtype=XXXXX | table txid]

Re: All results are not returned with multiple field exclusions

sarahalhawi — Tue, 29 Sep 2020 10:13:38 GMT

The query I posted is correct (i have just put XXX in the place of sensitive information).

I want to be able to do

EVT*-XXXXX[search eventtype=XXXXX | table XXXXX] vsp_vendor_id!=XXXXXX vsp_vendor_id!=XXXXXX host!=XXXXXX host!=XXXXXX

and get results for more than 2 days.

Re: All results are not returned with multiple field exclusions

somesoni2 — Wed, 13 Jul 2016 15:03:54 GMT

What's the purpose of subsearch? Are you using third filter based on some field?

Re: All results are not returned with multiple field exclusions

sarahalhawi — Wed, 13 Jul 2016 15:22:11 GMT

Yes, to exclude certain values.

Re: All results are not returned with multiple field exclusions

maciep — Wed, 13 Jul 2016 16:02:04 GMT

Have you tried inspecting the job to see the search Splunk actually ends up running after the subsearch is resolved? Is there anything specific about the results in the past 2 days that might also explain why those are the only returned, as opposed to search just randomly returning results only that old?

On a side note, are you sure you don't want AND instead of OR here?

 (host!=XXXX OR host!=XXXX)

It's hard to tell with the obfuscated data, but OR'ing together negatives doesn't seem to accomplish much. For example, if I have a host named server1, then (host!=server1 OR host!=server2) is still going to resolve to true because the second condition is true.

Re: All results are not returned with multiple field exclusions

sarahalhawi — Wed, 13 Jul 2016 16:10:47 GMT

Yes, if I run the query without the filters, EVT*-XXXX [search eventtype=XXXXX | table txid], I get results for more than 2 days.

Thanks for the tips on the OR, just using a list of values now rather than AND or OR.

Re: All results are not returned with multiple field exclusions

maciep — Wed, 13 Jul 2016 16:22:35 GMT

But for those results without the filters, are there older events that would match filter? I mean, you are filtering, so maybe there isn't older data that actually matches your filter. If there is older data that matches your filter, maybe there is typo in your filter like woodcock suggested.

We don't have your data so we can't do that troubleshooting, but you should be able to make your way through the results to figure out what's happening.

Also, if you haven't yet, inspect the job to see what Splunk actually searching after the subsearch is resolved. Maybe there will be something noticeable/obviously wrong if you look at it there.

Re: All results are not returned with multiple field exclusions

somesoni2 — Wed, 13 Jul 2016 16:28:29 GMT

Give this a try

EVT*-XXXX [search eventtype=XXXXX | stats count by txid | table txid ] NOT (host=XXXX OR host=XXXX OR "vsp-vendor-id=XXXXXXXXXXXXXX")

Re: All results are not returned with multiple field exclusions

sarahalhawi — Thu, 14 Jul 2016 08:52:19 GMT

thanks for that maciep, i ran the subsearch independently and older results are returned. Will give the job inspection a try 🙂

Re: All results are not returned with multiple field exclusions

sarahalhawi — Thu, 14 Jul 2016 12:36:24 GMT

thank you so much this has worked!! legend

Re: All results are not returned with multiple field exclusions

sarahalhawi — Thu, 14 Jul 2016 12:40:42 GMT

do you know why the quotes are required? without the quotes, the right results are not returned.