Is there a limit with having multiple "SOURCE !=" in a search?

[geelsu]

Newbie here. I was exploring Dashboard setup, so started doing some searches to create one with. I started eliminating Windows Events with the following:

source!="WinEventLog:Application" source!="WinEventLog:Security"  source!="WinEventLog:Setup"

Next I clicked on the the WinEventLog:System event to exclude it from the search, but it displayed 0 events in the little popup, but there are tons of them. When I added it to the search bar like this... :

source!="WinEventLog:Application" source!="WinEventLog:Security" source!="WinEventLog:Setup" source!="WinEventLog:System"

... no events displayed, but I know there are other events which I am trying to narrow down to. Why does adding a fourth Source!= result in the display of no results at all? It happens no matter what order the WinEventLog types are entered. I've tried rearranging them, but get the same results. Obviously, this may not be the best way to narrow down to a specific event, but this approach has me wondering what is wrong.

[esix_splunk]

You may want to look into the differences between the "!=" and NOT (XXX) searches. There are some intrinsic differences, and its probably why you are seeing some oddities..

As Lisa says, your search is inefficient... I would recommend adjusting..

index=myindex NOT (source="WinEventLog:Application" OR source="WinEventLog:Security"  OR source="WinEventLog:Setup")

Here is some more reading on similar topics:
https://answers.splunk.com/answers/43228/use-of-not-vs.html
http://docs.splunk.com/Documentation/Splunk/6.4.0/Search/Usethesearchcommand#Difference_between_NOT_...

[lguinn2]

You may be hitting some limits or quirks of the user interface or the browser, I am not sure.

However, I do know that this search is incredibly inefficient. First, the "*" searches across all indexes, sources, hosts and sourcetypes. Second, using "not equal" (or NOT in general) causes a slower search - Splunk can't use the keyword index effectively and can end up doing a serial scan of the data.

Sure, when you are doing forensics or tracking down an error, the "click" interface can make it easier to do your ad-hoc investigation.

Frankly, I would be tempted to run a search like this:

sourcetype=Win* 
| stats count by host source sourcetype
| sort 0 count

And then using drill-down to click in the search results for details. It's not terrifically efficient, but it gives you an overview. Perhaps it can tell you what to search "for", which is much more efficient than searching by elimination.

[Richfez]

I noticed some interesting behavior but it's not quite like yours. Below I try to reproduce what I'm doing in case it is related.

In the meantime, what browser, browser version, splunk version and edition are you using? I hate to ask, but have you tried rebooting your client, and possibly restarting Splunk on the server involved?

My behavior is transient and less of an issue. If I have multiple tabs open, I can sometimes create a situation where when I click "source=..." my drop down with "Exclude from search" or "Add to search" won't have the event count. I believe it happens about 1/3rd or 1/2 of the time when I switch to a different tab, switch back to Splunk and click. Excluding the selected one (the one currently without numbers) works fine, and attempts to follow this same procedure on subsequent event sources also works totally fine and normally (with numbers), so I think it may just be something with caching.

Just FYI I'm on Firefox latest on Windows 10, with in this case my test server being my W7 laptop running Splunk 6.4. In that setup I have managed to fill up my search entirely (33 sources excluded) for the past 24 hour period without any issues except that minor one I mention above.

My resulting, working search that finally on the last "exclusion" dropped me to a zero event count:

* source!="WinEventLog:System" source!="WinEventLog:Microsoft-Windows-Bits-Client/Operational" source!="WinEventLog:Microsoft-Windows-GroupPolicy/Operational" source!="WinEventLog:Microsoft-Windows-TaskScheduler/Operational" source!="WinEventLog:Security" source!="WinEventLog:Microsoft-Windows-WindowsUpdateClient/Operational" source!="WinEventLog:Microsoft-Windows-Diagnosis-DPS/Operational" source!="WinEventLog:Application" source!="WinEventLog:Microsoft-Windows-Windows Firewall With Advanced Security/Firewall" source!="WinEventLog:Microsoft-Windows-Dhcpv6-Client/Admin" source!="WinEventLog:Cisco AnyConnect Secure Mobility Client" source!="WinEventLog:Microsoft-Windows-NetworkProfile/Operational" source!="WinEventLog:Microsoft-Windows-Diagnostics-Performance/Operational" source!="WinEventLog:Microsoft-Windows-Dhcp-Client/Admin" source!="WinEventLog:Microsoft-Windows-WLAN-AutoConfig/Operational" source!="WinEventLog:Microsoft-Windows-WindowsBackup/ActionCenter" source!="WinEventLog:Microsoft-Windows-DriverFrameworks-UserMode/Operational" source!="WinEventLog:Microsoft-Windows-Application-Experience/Program-Telemetry" source!="WinEventLog:Microsoft-Windows-LanguagePackSetup/Operational" source!="WinEventLog:Microsoft-Windows-MUI/Operational" source!="WinEventLog:Microsoft-Windows-ReliabilityAnalysisComponent/Operational" source!="WinEventLog:Microsoft-Windows-Known Folders API Service" source!="WinEventLog:Microsoft-Windows-WinRM/Operational" source!="WinEventLog:Microsoft-Windows-HomeGroup Provider Service/Operational" source!="WinEventLog:Microsoft-Windows-TerminalServices-LocalSessionManager/Operational" source!="WinEventLog:Microsoft-Windows-User Profile Service/Operational" source!="WinEventLog:Microsoft-Windows-Resource-Exhaustion-Detector/Operational" source!="WinEventLog:Setup" source!="WinEventLog:Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational" source!="WinEventLog:Microsoft-Windows-Kernel-WHEA/Operational" source!="WinEventLog:Microsoft-Windows-NlaSvc/Operational" source!="WinEventLog:Microsoft-Windows-Resource-Exhaustion-Resolver/Operational"