I am using the following to extract two fields at search time, extract_domain and extract_ip
source="dns2.log"
| rex "((?<extract_domain>(\w+(\(\d\))){1,}?)$)"
| rex mode=sed field=extract_domain "s/(\\(\\d\\))/./g"
| rex "(?<extract_ip>\b(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b)"
| rex mode=sed field=extract_domain "s/.$//"
I would like to lookup the extract_domain
field against a .csv file which looks something like this
domain, status
splunk.com, good
facebook.com,bad
google.com, good
hi5.com, bad
I want to report on all domains with a status of bad.
I've followed the example for using lookup tables but it isnt working out.
Please help
I suspect the issue is the order in which splunk is performing the lookup vs field extractions. Since you are extracting fields in your search explicitly, then you will want to use the lookup
command explicitly after your fields have been extracted via the rex
commands. Otherwise, I believe that splunk attempt to do automatic lookups immediately after your base search (which is simply source="dns2.log"
, at which point in, extract_domain
does not yet exist.)
I would suggest you try:
<your existing search> | lookup domainstatus domain OUTPUT status
Once you get this working (assuming you want this to be done for other searches too) then I would suggest moving your field extractions to configuration files and then enable an automatic lookup for your source (or more preferably, for your specific sourcetype.)
I'm struggling to find the exact search time operation order in the docs, or I'd link to it here.
What isn't working out? Lookup tables would indeed be ideal to use here.
Put your csv file in a "lookups" directory, for instance $SPLUNK_HOME/etc/system/lookups
. On the first line, put the field names you want to use, for instance "domain,status". Call the file something, say, "domainstatus.csv".
Then refer to it in transforms.conf:
[domainstatus]
filename = domainstatus.csv
Finally refer to the lookup transform in props.conf:
[source::dns2.log]
LOOKUP-domainstatus = domainstatus domain OUTPUT status
After restarting Splunk you should now see a field "status" for each "domain" for the source "dns2.log".