topic Re: CSV and InputLookup in Getting Data In

CSV and InputLookup

SamHarris — Mon, 28 Sep 2020 13:05:32 GMT

Ok,

I am sure there is something here about this already, but I have looked for it and just can not seem to find it.

I am looking to run a search on a csv file that is populated with web addresses and IP Addresses to see if any we have hit in of these sites. I have the CSV file set in UTF-8 coding and looks like this:

Websites
"Http://www.google.com"
"Http://www.yahoo.com"
"10.1.1.1"

Now I want to compare this to a sourtype called Gateway and have tried to following search and can't seem to get any results (even though I search for the website without the inputlookup command and it is triggered)

sourcetype=gateway | inlookup Websites.CSV | stats sparkline count values(src_ip) as src_ip by domain

Any help would be appericiated! Thank you!

Re: CSV and InputLookup

Ayn — Fri, 11 Jan 2013 15:25:32 GMT

inputlookup needs to be the first command in a search. While you won't get a syntax error in other cases, it won't make sense to use it then.

If I understand your need correctly, you have data with sourcetype "Gateway" and in this data you want to look for strings that come from the csv file Websites.CSV. You can do this with a subsearch. Like this:

sourcetype=gateway [| inputlookup Websites.CSV | return $Websites] | stats sparkline count values(src_ip) as src_ip by domain

This will have the effect that the original search looks something like this after the subsearch is done:

sourcetype=gateway ("Http://www.google.com" OR "Http://www.yahoo.com" OR "10.1.1.1") | ...

Re: CSV and InputLookup

SamHarris — Fri, 11 Jan 2013 16:07:20 GMT

Think I still might not be getting it...

When I run your command into our search I get populated with EVERYTHING everyone hits within the last 15 minutes. I only would like to see the websites and IPs that we hit within the CSV, but I will also not rule out that my CSV and lookup fields are incorrect as well.

Re: CSV and InputLookup

Ayn — Fri, 11 Jan 2013 16:11:38 GMT

That doesn't sound right, unless you have things in Websites.CSV that will match everything. You can check the resulting search string by running a variant of the subsearch on its own and adding | format at the end:

| inputlookup Websites.CSV | rename Websites as query | format

This is the filter that the main search will use. If it includes terms that will function as catch-all filters, then there's your problem.