Why is my inputlookup search not returning all results from the CSV file?

[jbradley76]

Hello All,

I have a CSV file with a single column, address, with about 1000 IP addresses below. I am attempting to search our proxy logs for any of the IP addresses that have accessed the internet and display the IP address with the TLD's that they have visited.

If I do a simple |inputlookup iplisttest.csv I receive the output of the CSV that lists all the IP's perfectly.

address
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4

If I do a full search of sourcetype="proxy" [|inputlookup iplisttest.csv | return $address] | stats values(dest) by src I get the first result only which is actually the second IP address in the list.

src                values(dest)
2.2.2.2             website2.com

Been beating my head against the wall on this one, so any help out be greatly appreciated!

[jbradley76 · Accepted solution]

index=idx_proxy sourcetype="proxy" 
| fields dest, src 
| lookup iplisttest.csv address as src OUTPUT address as src_ip | search src_ip=* | stats values(dest) by src

This is what solved the problem, found by one of our engineers. Seems the lookup command did the trick!

Thanks All!

[somesoni2]

Try something like this (assuming you want to do a text search on the IP addresses from the lookup)

sourcetype="proxy" [|inputlookup iplisttest.csv | eval search=address | table search] | stats values(dest) by src

[Lowell]

Subsearches will only return up to 10,000 results by default. Check out limits.conf docs for more details.

You may want to consider rewriting the search to avoid using a subsearch by using the lookup command directly.