Getting Data In

How to search for events which match entry in a CSV whilst also discounting entries from a seperate CSV?

adamhands96
Engager

Hello wonderful people of the internet,

I'm still quite new when it comes to using splunk, so could use a bit of advice with this one. I have 2 CSV files, both containing a list of IP addresses. One of these is called IOC1.csv, and is a file of known malicious addresses.
The second CSV, called ignore.csv, contains all of the IP addresses I wish to exclude from the results (Basically, stuff we want to discount/tune out).

I'd like a search which could check all of the FW logs for any hits which have an IP from IOC1.csv located in there, but discount the event if an IP from ignore.csv is also present.

Could somebody advise on how this could be done?

Thank you all so much.

0 Karma
1 Solution

gcusello
Esteemed Legend

Hi @adamhands96,
let me know: you want to match your events with the first lookup and exclude from the results the IPs of the second lookup, is it correct?
But, is it possible that some IPs of the second list are in the first? how is it possible?

Anyway, you have to run something like this:
if you have a field called IP in your events:

index=my_index [ | inputlookup IOC1.csv | fields IP ] NOT [ | inputlookup IOC2.csv | fields IP ] 
| ...

if instead, you haven't a field containing IP, you have to run:

index=my_index [ | inputlookup IOC1.csv | rename IP AS query | fields query ] NOT [ | inputlookup IOC2.csv | rename IP AS query | fields query ] 
| ...

Ciao.
Giuseppe

View solution in original post

0 Karma

gcusello
Esteemed Legend

Hi @adamhands96,
let me know: you want to match your events with the first lookup and exclude from the results the IPs of the second lookup, is it correct?
But, is it possible that some IPs of the second list are in the first? how is it possible?

Anyway, you have to run something like this:
if you have a field called IP in your events:

index=my_index [ | inputlookup IOC1.csv | fields IP ] NOT [ | inputlookup IOC2.csv | fields IP ] 
| ...

if instead, you haven't a field containing IP, you have to run:

index=my_index [ | inputlookup IOC1.csv | rename IP AS query | fields query ] NOT [ | inputlookup IOC2.csv | rename IP AS query | fields query ] 
| ...

Ciao.
Giuseppe

0 Karma

adamhands96
Engager

you want to match your events with the first lookup and exclude from the results the IPs of the second lookup, is it correct?
Correct ..

But, is it possible that some IPs of the second list are in the first? how is it possible?
It's possible, yes. If that was the case I'd like the event to simply be removed from the results.

Thanks for your above answer - I have 2 fields within my results that i'd need searching, src_ip & dest_ip. Is that going to be possible, or would it be easier to have a seperate search for each?

Thank you so much.

0 Karma

gcusello
Esteemed Legend

Hi @adamhands96,
no you can run a single search:

 index=my_index ([ | inputlookup IOC1.csv | rename IP AS src_ip | fields src_ip  ] OR [ | inputlookup IOC1.csv | rename Ip AS dest_ip | fields dest_ip ])  NOT ([ | inputlookup IOC2.csv | rename IP AS src_ip | fields src_ip  ] OR [ | inputlookup IOC2.csv | rename Ip AS dest_ip | fields dest_ip ])
| ...

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Platform Newsletter Highlights | March 2023

 March 2023 | Check out the latest and greatestIntroducing Splunk Edge Processor, simplified data ...

Enterprise Security Content Updates (ESCU) - New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 3 releases of new content via the Enterprise ...

Thought Leaders are Validating Your Hard Work and Training Rigor

As a Splunk enthusiast and member of the Splunk Community, you are one of thousands who recognize the value of ...