How to match fields from indexed data with CSV lookup

[jon3484]

I created a list of known malicious domain names and put that information into a CSV. I named the field "dest_hostname", the same as what it shown in the firewall logs.

Ex:

Field name: dest_hostname
Field values: 2030049929xxxuuu.com, somewhere.ru, 019293fsvs.br, 120344fruexe.com etc.

I want to know if there is a way that you can search in Splunk for this information on the firewall to see if there is a match with one of these domain names from my CSV file.

Ex.

Oct 21 10:57:30 STSFW01.XYZ.com 1,2017/10/24 10:57:30,002201000314,THREAT,url,0,2017/10/24 10:57:30,10.8.0.56,23.21.144.160,0.0.0.0,0.0.0.0,Exempted_Applications,,,ssl,vsys1,trust-L2,untrusted-L2,ethernet1/22,ethernet1/21,Border_FWD,2017/10/24 10:57:30,34984390,1,51716,443,0,0,0xb000,tcp,alert,"019293fsvs.br",(9999),ssl-decryption-issues,informational,client-to-server,11081430324,0x0,10.0.0.0-10.255.255.255,US,0,,0,,,0,,,,,,,,0,17,0,0,0,

Splunk would generate an alert because 019293fsvs.br domain is in my CSV. Then table that information with the fields from the actually firewall log, so "time, src, dst, dest_port url, count" etc.

Thanks, I'm still new to lookups and Splunk in general.

[Genti · Accepted solution]

have your lookup be a 2 column table, the first column will be your dest_hostname, the second will be any name you chose. I would say since you are either trying to create an alert, call it "alert", or since it is as you say a malware site, call it "malware_site".
the values underneath the "alert" or "malware_site" should be 1, for each of your dest_hostname

so, it should look something like this:

dest_hostname, malware_site 
2030049929xxxuuu.com, 1 
somewhere.ru, 1 
019293fsvs.br, 1 
120344fruexe.com, 1 
etc...,1

The second step is to ensure that your dest_hostname is indeed an extracted field in your data, ie. your log that looks like so:

Oct 21 10:57:30 STSFW01.XYZ.com 1,2017/10/24 10:57:30,002201000314,THREAT,url,0,2017/10/24 10:57:30,10.8.0.56,23.21.144.160,0.0.0.0,0.0.0.0,Exempted_Applications,,,ssl,vsys1,trust-L2,untrusted-L2,ethernet1/22,ethernet1/21,Border_FWD,2017/10/24 10:57:30,34984390,1,51716,443,0,0,0xb000,tcp,alert,"019293fsvs.br",(9999),ssl-decryption-issues,informational,client-to-server,11081430324,0x0,10.0.0.0-10.255.255.255,US,0,,0,,,0,,,,,,,,0,17,0,0,0,

has a correct field extraction that captures dest_host=019293fsvs.br

Assuming that the above are done, you then can run a search like:

<your search here> | lookup <your lookup file> dest_hostname OUTPUT malware_site | search malware_site=1 | table "time, src, dst, dest_port url, count" etc...

Save this as an alert to notify if count>0, and enjoy the email you receive with the alert.
cheers,
.gz

[jon3484]

Thanks! 🙂