topic Re: Using lookup in Getting Data In

Using lookup

chrispounds — Thu, 30 May 2019 11:41:10 GMT

Hi All,

So i have a csv, which only contains one field "ip" with ips listed.

I would like to inputlookup this csv into a search something like below

index=firewall src_ip=(list of ip's from csv) | *rest of search*

i know you have to do [inputlookup ip_list.csv | fields ip] to get the csv loaded, i'm just not sure how to then adapt it to the original search criteria, could any of you smart people help?

Many thanks,

Re: Using lookup

DavidHourani — Thu, 30 May 2019 11:53:06 GMT

Hi @chrispounds,

You're looking for something like this :

index=firewall  [inputlookup ip_list.csv|rename ip as src_ip | return 100 src_ip] | rest of search

Cheers,
David

Re: Using lookup

jgoodrich — Wed, 30 Sep 2020 00:44:10 GMT

I would recommend changing your csv to have two columns. For example "lookup_ip" and "lookup_reason". The second column could just contain a single value in each row that describes what the list is for (i.e. "my ip blacklist"). Then add the lookup to Splunk. For your search do:

index=firewall
| lookup ip_list.csv lookup_ip as src_ip

In the results, if any event has a src_ip that matches the lookup list you'll see the "lookup_reason" field show up. So you'll have lookup_reason="my ip blacklist". This is a super easy way to pull lookup data in, then you can work it downstream in the search pipeline.

Re: Using lookup

ddrillic — Thu, 30 May 2019 13:44:37 GMT

For reference @chrispounds, this beautiful answer by @acharlieh at What is the basic difference between the lookup, inputlook and outputlookup commands

Re: Using lookup

chrispounds — Wed, 30 Sep 2020 00:45:43 GMT

Hi @DavidHourani ,

So one of the interesting fields is src_ip already from the firewall log. How would i link the "ip" field from the ip_list.csv, to the actual field?

In essence, using the "ip" field in the lookup as a list of IP's to look for like its an OR statement, does that make sense?

Many thanks,
Chris

Re: Using lookup

chrispounds — Thu, 30 May 2019 14:42:27 GMT

thanks @ddrillic i'll definitely check that out! 😄

Re: Using lookup

DavidHourani — Wed, 30 Sep 2020 00:45:48 GMT

Hey @chrispounds,

This [inputlookup ip_list.csv|rename ip as src_ip | return 100 src_ip] in the search above allows u to rename the field from the lookup to src_ip and will automatically filter on those IPs only. Try using [inputlookup ip_list.csv|rename ip as src_ip | return 100 src_ip] alone and see what it returns, it will be the list of IPs seperated by OR.

If you want to enrich your data with the lookup then you can use something like this :

 index=firewall | lookup ip_list.csv ip AS src_ip