How to search for events which match entry in a CSV whilst also discounting entries from a seperate CSV?

[adamhands96]

Hello wonderful people of the internet,

I'm still quite new when it comes to using splunk, so could use a bit of advice with this one. I have 2 CSV files, both containing a list of IP addresses. One of these is called IOC1.csv, and is a file of known malicious addresses.
The second CSV, called ignore.csv, contains all of the IP addresses I wish to exclude from the results (Basically, stuff we want to discount/tune out).

I'd like a search which could check all of the FW logs for any hits which have an IP from IOC1.csv located in there, but discount the event if an IP from ignore.csv is also present.

Could somebody advise on how this could be done?

Thank you all so much.

[gcusello · Accepted solution]

Hi @adamhands96,
let me know: you want to match your events with the first lookup and exclude from the results the IPs of the second lookup, is it correct?
But, is it possible that some IPs of the second list are in the first? how is it possible?

Anyway, you have to run something like this:
if you have a field called IP in your events:

index=my_index [ | inputlookup IOC1.csv | fields IP ] NOT [ | inputlookup IOC2.csv | fields IP ] 
| ...

if instead, you haven't a field containing IP, you have to run:

index=my_index [ | inputlookup IOC1.csv | rename IP AS query | fields query ] NOT [ | inputlookup IOC2.csv | rename IP AS query | fields query ] 
| ...

Ciao.
Giuseppe

[adamhands96]

you want to match your events with the first lookup and exclude from the results the IPs of the second lookup, is it correct?
Correct ..

But, is it possible that some IPs of the second list are in the first? how is it possible?
It's possible, yes. If that was the case I'd like the event to simply be removed from the results.

Thanks for your above answer - I have 2 fields within my results that i'd need searching, src_ip & dest_ip. Is that going to be possible, or would it be easier to have a seperate search for each?

Thank you so much.

[gcusello]

Hi @adamhands96,
no you can run a single search:

 index=my_index ([ | inputlookup IOC1.csv | rename IP AS src_ip | fields src_ip  ] OR [ | inputlookup IOC1.csv | rename Ip AS dest_ip | fields dest_ip ])  NOT ([ | inputlookup IOC2.csv | rename IP AS src_ip | fields src_ip  ] OR [ | inputlookup IOC2.csv | rename Ip AS dest_ip | fields dest_ip ])
| ...

Ciao.
Giuseppe