I have a dashboard which displays some simple "top 15" visualizations based on outbound network traffic. The base search just pulls some basic stats from All_Traffic, filtering in the tstats ... whe...
See more...
I have a dashboard which displays some simple "top 15" visualizations based on outbound network traffic. The base search just pulls some basic stats from All_Traffic, filtering in the tstats ... where clause to include only outbound traffic. I define "outbound" to be any traffic for which the source is an internal IP and the destination is NOT an internal IP.
This worked up until we upgraded from to Splunk 7.3.1 to 8.0.1, but now the clause filtering out All_Traffic.dest_ip!=10.0.0.0/8 , etc. are completely ignored (running the same search with and without the condition return the same results without the desired filtering)
Here's the original base search:
| tstats count(All_Traffic.dest_ip) AS ip_count count(All_Traffic.dest_port) AS port_count from datamodel=Network_Traffic where (All_Traffic.src_ip=10.0.0.0/8 OR All_Traffic.src_ip=192.168.0.0/16 OR All_Traffic.src_ip=172.16.0.0/12) AND NOT (All_Traffic.dest_ip=10.0.0.0/8 OR All_Traffic.dest_ip=192.168.0.0/16 OR All_Traffic.dest_ip=172.16.0.0/12) by All_Traffic.dest_ip, All_Traffic.dest_port
| rename All_Traffic.* AS *
A simpler version with only one exclusion in the tstats ... where clause which also does not work:
| tstats count(All_Traffic.dest_ip) AS ip_count count(All_Traffic.dest_port) AS port_count from datamodel=Network_Traffic where All_Traffic.dest_ip!=10.0.0.0/8 by All_Traffic.dest_ip, All_Traffic.dest_port
| rename All_Traffic.* AS *
This seems very similar (but not identical) to the problem described in the release notes for 8.0.1 as fixed:
SPL-179594, SPL-177665 - tstats where clause does not filter as expected when structured like "WHERE * NOT (field1=foo AND field2=bar)"*
Also seems related to the question here: hxxps://answers.splunk.com/answers/760542/why-only-one-condition-works-for-where-clause-in-a.html
Similar to the asker above, I am hoping to do the filtering in the WHERE clause of the tstats for performance. I run this search over the past 24h and it takes a while to run. I'd rather not split the tstats by src_ip and have to reaggregate with another stats, and would prefer to do the filtering BEFORE passing the stats to |search.
I can work around it if I have to (the search below DOES work), but I'd rather go with something a bit more performant.
| tstats count(All_Traffic.dest_ip) AS ip_count count(All_Traffic.dest_port) AS port_count from datamodel=Network_Traffic by All_Traffic.dest_ip, All_Traffic.dest_port, All_Traffic.src_ip
| rename All_Traffic.* AS *
| where (cidrmatch("10.0.0.0/8",src_ip) OR cidrmatch("172.16.0.0/12",src_ip) OR cidrmatch("192.168.0.0/16",src_ip) OR cidrmatch("169.254.0.0/16",src_ip)) AND NOT (cidrmatch("10.0.0.0/8",dest_ip) OR cidrmatch("172.16.0.0/12",dest_ip) OR cidrmatch("192.168.0.0/16",dest_ip) OR cidrmatch("169.254.0.0/16",dest_ip))