I'd like to use a lookup list of known bad domains to compare againt my DNS logs, but I'm not sure how to do a substring search in parallel with a lookup.
For example, my csv looks like this:
bad_domain
domain.com.
domain.co.uk.
sub.malware.com.
However, the actual query in my logs could be string.domain.com., something.interesting.domain.co.uk., etc..
Is there some way to use the lookup list just as the "right-most" part of the lookup. I can get EXACT matches doing this pretty easily:
| join max=0 query
[| inputlookup domain-blacklist
| rename bad_domain as query
| fields + query ]
Thanks
Don't do a join. Just use the lookup. Either make the lookup a wildcard lookup or use a regex to field extract off the part of the domain you want and have that match what is in your lookup.
search... | lookup domain-blacklist bad_domain as query_domain OUTPUTNEW bad_domain as domainIsBad | where isnotnull(domainIsBad)
Don't do a join. Just use the lookup. Either make the lookup a wildcard lookup or use a regex to field extract off the part of the domain you want and have that match what is in your lookup.
search... | lookup domain-blacklist bad_domain as query_domain OUTPUTNEW bad_domain as domainIsBad | where isnotnull(domainIsBad)
Thanks a lot for this - it resolved my problem.
I wasn't aware of the wildcard and that completely solved my problem. The link you provided allowed me to also find this article which gave me a good sample how to do this in practice: https://answers.splunk.com/answers/52580/can-we-use-wildcard-characters-in-a-lookup-table.html
After adding *. in front of all the domain names in my lookup, I was able to get this working.
Note: my DEV Splunk running 6.6.4 has the match_type available in the GUI, but my prod instance (6.5.0) didn't, but manually editing the transforms.conf per the above article works.