topic Re: searching across multiple indexes with lookups in Splunk Search

searching across multiple indexes with lookups

caseysutherland — Wed, 20 Dec 2017 16:17:05 GMT

we have two indexes with some overlap in fields. specifically IP addresses. what I would like to is do an initial search dedup all the dest_ips then in the same search string use the deduped list of IPs as the filter on the second index. Is this possible? I would think some type of lookup table is the way to go.

Re: searching across multiple indexes with lookups

elliotproebstel — Wed, 20 Dec 2017 16:28:31 GMT

It depends on the size of the result set and the frequency with which you'll run this. If you want to save the results of that first deduped query to use over and over, the lookup file would be a good way to go:

index=first_index
| stats values(dest_ip) AS dest_ip
| outputlookup ip_list

And then to use it in your subsequent searches:

index=second_index 
[ | inputlookup ip_list 
  | stats values(dest_ip) AS dest_ip 
  | format ]

If it's just an ad-hoc thing you want to run once, you are likely fine with a single subsearch:

index=second_index
[ search index=first_index
  | stats values(dest_ip) AS dest_ip
  | format ]

Re: searching across multiple indexes with lookups

harsmarvania57 — Wed, 20 Dec 2017 16:28:48 GMT

Hi,

Can you please try something like this?

index=<index 1> [search index=<index 2> | dedup dest_ip| return dest_ip]

Re: searching across multiple indexes with lookups

caseysutherland — Tue, 29 Sep 2020 17:20:04 GMT

does this return all the events in index one that have a dest_ip that exists in the list of dest_ips returned from index 2?

Re: searching across multiple indexes with lookups

harsmarvania57 — Tue, 29 Sep 2020 17:19:01 GMT

This will display dest_ip which are in Index 2 and then AND with index 1 dest_ip

So this query works like this index= index 1 AND dest_IP from Index 2