Splunk Search

How to optimize my search to find 30+ src_ips and any dest_ip that are not 2 specific IP addresses?

packet_hunter
Contributor

Currently I am using (OR)s

For example:

Index = A sourcetype=a (src="192.168.3.5" OR src="192.168.3.6" OR.... etc.) | stats list(dest_ip) by src_ip

Currently I filter thru the dest_ip list, but I would like to add a NOT dest_ip= 10.10.3.5 OR NOT dest_ip=10.10.3.6

Any advice to optimize and speed up the search is greatly appreciated!

Thank you

Tags (3)
0 Karma
1 Solution

sundareshr
Legend
0 Karma

stevejfice
Path Finder

It might depend on how your data is being indexed but with my Palo Alto data I can do the following:

index=pan src_ip<192.168.2.40 src_ip>192.168.2.20 | stats list(dest_ip) by src_ip

(obviously my source ip's are different but the syntax works).

Interestingly if I switch the two src_ip searches around (greater than first) it fails to work, not sure why but I'm sure a more intelligent person than me can find out)

0 Karma

packet_hunter
Contributor

Thank you for the reply and interesting observation.
My problem is that I often need to search a number of internet src IPs that are unrelated but I will try ordering them and see it that does anything. My primary effort is too speed up the search process without summary indexing b/c the IPs of concern change frequently.

Thank you again for your reply, I will let you know the results of my testing if there is anything identifiable.

0 Karma

sundareshr
Legend
0 Karma

packet_hunter
Contributor

the IP lists change often, does an inputlookup speed things up?

0 Karma

sundareshr
Legend

inputlookup will not only just create the same "OR" search, it will include a sub-search (extra step). Don't see that being faster.

Look at "Common causes of slow searches" section

http://wiki.splunk.com/Community:PerformanceTroubleshooting

0 Karma

packet_hunter
Contributor

I have been looking into Summary indexing. Please make summary index your answer and I will accept.

Thank you

0 Karma

packet_hunter
Contributor

Ok, I follow what you are saying, thank you

0 Karma

sundareshr
Legend

See if this works (haven't tried it)

index=a sourcetype=b NOT (dest_ip=10.10.3.5 OR dest_ip=10.10.3.6) | regex src="([0-9]{2})$"
0 Karma

packet_hunter
Contributor

Do you have any suggestions for speeding up a search with >30 IPs: e.g. src="" OR src="" etc

I have been considering the use of a lookup table (with all the IPs), but not sure if that will speed the search up.

I am searching thru firewall logs and my time frame is last 30 days, but it always times out when I use all the OR(s).

Thank you

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...