Currently I am using (OR)s
For example:
Index = A sourcetype=a (src="192.168.3.5" OR src="192.168.3.6" OR.... etc.) | stats list(dest_ip) by src_ip
Currently I filter thru the dest_ip list, but I would like to add a NOT dest_ip= 10.10.3.5 OR NOT dest_ip=10.10.3.6
Any advice to optimize and speed up the search is greatly appreciated!
Thank you
Summary Index is probably your best bet.
http://docs.splunk.com/Documentation/Splunk/6.4.2/Knowledge/Usesummaryindexing
It might depend on how your data is being indexed but with my Palo Alto data I can do the following:
index=pan src_ip<192.168.2.40 src_ip>192.168.2.20 | stats list(dest_ip) by src_ip
(obviously my source ip's are different but the syntax works).
Interestingly if I switch the two src_ip searches around (greater than first) it fails to work, not sure why but I'm sure a more intelligent person than me can find out)
Thank you for the reply and interesting observation.
My problem is that I often need to search a number of internet src IPs that are unrelated but I will try ordering them and see it that does anything. My primary effort is too speed up the search process without summary indexing b/c the IPs of concern change frequently.
Thank you again for your reply, I will let you know the results of my testing if there is anything identifiable.
Summary Index is probably your best bet.
http://docs.splunk.com/Documentation/Splunk/6.4.2/Knowledge/Usesummaryindexing
the IP lists change often, does an inputlookup speed things up?
inputlookup will not only just create the same "OR" search, it will include a sub-search (extra step). Don't see that being faster.
Look at "Common causes of slow searches" section
I have been looking into Summary indexing. Please make summary index your answer and I will accept.
Thank you
Ok, I follow what you are saying, thank you
See if this works (haven't tried it)
index=a sourcetype=b NOT (dest_ip=10.10.3.5 OR dest_ip=10.10.3.6) | regex src="([0-9]{2})$"
Do you have any suggestions for speeding up a search with >30 IPs: e.g. src="" OR src="" etc
I have been considering the use of a lookup table (with all the IPs), but not sure if that will speed the search up.
I am searching thru firewall logs and my time frame is last 30 days, but it always times out when I use all the OR(s).
Thank you