topic Re: Search incorporating inputlookup in Splunk Search

Search incorporating inputlookup

balcv — Mon, 12 Apr 2021 23:58:40 GMT

I have a list of source ip addresses in a csv file loaded into Splunk as a lookup file. The file has a single field, src_ip, and about 4000 rows of unique ip address.

I want to take the contents of the lookup file and compare each entry to a search of filewall logs and report the number of times each entry in the lookup file is present in the firewall data.

I have this so far but the src_ip listed in the result is not always present in the lookup file.

Any suggestions greatly appreciated.

Thanks
Leigh

Re: Search incorporating inputlookup

96nick — Tue, 13 Apr 2021 00:09:17 GMT

Try this one out:

Similar to the linked reply below. Does that get you what you need? Hope this helped!

Solved: Search for all events for IP address within a CSV ... - Splunk Community

Re: Search incorporating inputlookup

richgalloway — Thu, 15 Apr 2021 13:05:21 GMT

That query takes all of the src_ip values found in the firewall index and adds to it all of the src_ip values from the RYUK lookup file. The result is the union, rather than the intersection, of the two sets.

To get the intersection, use this query to tell Splunk to only pull events that contain a listed src_ip value.

index="firewall" src_ip!="192.168.0.0/16" [ | inputlookup RYUK.csv | return 1000 src_ip] | stats count by src_ip

The subsearch reads the lookup file and reformats it into a query string of the form "(src_ip=1.2.3.4 OR src_ip=1.2.3.5 OR src_ip=1.3.4.5 ...)"

Re: Search incorporating inputlookup

balcv — Tue, 13 Apr 2021 01:29:29 GMT

Thanks @richgalloway That appears to have done the trick but I'll need to wait until I have data that matches, but I think it should be fine.

Thanks
Leigh

Re: Search incorporating inputlookup

balcv — Wed, 14 Apr 2021 23:49:41 GMT

Hi @richgalloway . Having had this running for a couple of days, it would appear that this search only takes the first address in the lookup file. I tested this by manually adding addresses I knew were valid and each time it only ever returned a count for the first address in the file and a count of that address.

It did not move through the entire lookup file.

Cheers
Leigh

Re: Search incorporating inputlookup

balcv — Wed, 14 Apr 2021 23:51:57 GMT

Hi @96nick . Thanks for your repsonse.

This it not giving me exactly what I wanted. It appears that it lists all the addresses found in the firewall log with a count rather than using the addresses in the lookup file.

I tested this by running the search then manually looking then up in the file lookup file and most did not exist in the file.

I need to take each address in the lookup file, compare to the firewall data and give a count for the number of matches against that file.

Cheers
Leigh

Re: Search incorporating inputlookup

richgalloway — Thu, 15 Apr 2021 13:05:59 GMT

You're right. I left out an argument to the return command. See the revised answer.