Block IP addresses by creating lookup file

[Karthikeya]

Hello,

We have a field called client_ip which contains different IP addresses and in events different threat messages will be there. 

So the ask is they want to exclude these IP addresses which contains threat messages. IPs are dynamic (different IPs daily) and threat messages also dynamic (different).

Normally to exclude this we need to give NOT (IP) NOT (IP)..... But here there are 100s of IPs and it will be big query.

What can be done in this case?

My thoughts..

Can I create a lookup table and user manually update that on daily basis and to exclude the IP addresses which are present in this lookup? Like just NOT (lookup table name) 

If it is good please help me with the workaround and query to be followed? 

Thanks in advance.

[Karthikeya]

Hi @gcusello ,

How this query can help me in excluding events of that particular IP addresses which have threat messages? In lookup table ip address will be filled manually daily or weekly basis by user.

But those IPs are to be excluded from search. I am confused. Please help me with the relevant query.

[gcusello]

Hi @Karthikeya ,

this is an inclusion condition, to have an exclusion condition you need only to add NOT before the subsearch

<your_search> NOT [ | inputlookup whitelisted_ips.csv | fields IP ]
| ...

or

<your_search> NOT [ | inputlookup whitelisted_ips.csv | rename ip AS query | fields query ]
| ...

You can use this search to exclude from your messages the IPs from your lookup.

If you want the search to automatically pole the lookup (if possible), I cannot help you because I don't know your data: you have to create a search that extract the IPs list, save it in the lookup and schedule it, something like this:

<your_search>
| dedup IP
| table IP
| outputlookup whitelisted_ips.csv

 Ciao.

Giuseppe

[isoutamo]

One comment what could helps you with ip based lookups.

When you are creating lookups which contains lookup and you need to find something there, you should/could use CIDR based searches. When you create lookup just define that it contains IP and search method is CIDR base.

One example https://community.splunk.com/t5/Splunk-Search/Using-CIDR-in-a-lookup-table/m-p/35787

[Karthikeya]

Hi @gcusello ,

Can you please guide me how to achieve this in brief? So that it will be helpful for me? What splunk query included in it?

[gcusello]

Hi @Karthikeya ,

follow these few steps:

• create a lookup called as you prefer (e.g. whitelisted_ips.csv),
• create a lookup definition with the same name adding in Advanced options CIDR in match_Type,
• Create a search that extract the IPs to whitelist,
• at the end of the search add the command | outputlookup whitelisted_ips.csv,
• using this search create an alert scheduling it with the frequency you like (e.g. once a day in the night)

in this way, you can use the lookup to exclude the IPS from your results running a search like the following (if the field in the lookup is "ip" and you want to search the ips in a field called ip):

<your_search> [ | inputlookup whitelisted_ips.csv | fields ip ]
| ...

if instead you want to search the ips in all the event, you can run:

<your_search> [ | inputlookup whitelisted_ips.csv | rename ip AS query | fields query ]
| ...

in this way, you execute a full text search on the _raw of your events.

Ciao.

Giuseppe

[gcusello]

Hi @Karthikeya ,

a lookup is surely a good solution!

I don't know if it's possible to extract with a search the IPs to be inserted in this lookup, if it's possible, you can create a search to extract these IPs and save them in the lookup using outputlookup, then , you can schedule this search to run e.g. once a day.

Otherwise, you can manage these list using the Lookup Editor App.

Rememeber, when you create this lookup to create the Lookup Definition and in it enable Match_Type CIDR (in Advanced options) so you can use range of IPs, so you don't need LIKE.

Ciao.

Giuseppe