I have a large lookup full of bad domains. The lookup is simply a domain name per line. I would like to search through a section of logs and return events where the bad domain exists. My domain list looks like
baddomain
baddomain1.com
baddomain2.com
...
And the query I'm attempting to write looks like
index=myindex | append [ inputlookup domains.csv | fields baddomain ] | where domain=baddomain | ...
If I run just the inputlookup portion of the query I'm seeing my results, but they do not seem to be getting appended.
Thanks, Kevin
How about:
index=myindex [ inputlookup domains.csv | rename baddomain as domain | fields baddomain]
This should be equivalent to:
index=myindex (domain=badguy.com OR domain=donkey.com ...)
So we worked the issue. The confusion was in the distributed search aspect.
We originally had the lookup file pushed out using the deployment server. So each indexer had the file in (app)/local/lookups. The search head did not have the lookup file.
It appears (please correct if this is wrong) that when using inputlookup, the lookup file need only be on the search head.
We were confusing this with regular lookups, where 'local=t' can be specified.
So we worked the issue. The confusion was in the distributed search aspect.
We originally had the lookup file pushed out using the deployment server. So each indexer had the file in (app)/local/lookups. The search head did not have the lookup file.
It appears (please correct if this is wrong) that when using inputlookup, the lookup file need only be on the search head.
We were confusing this with regular lookups, where 'local=t' can be specified.
GO TEAM NOVII!!
you could also try:
[] LOOKUP-baddomain = ..... OUTPUTNEW ...
in your props.conf using the true and is_bad field.
with associated entry in transforms.conf:
[baddomain] default_match = no_match filename = lookup_baddomains.csv max_matches = 1 min_matches = 1
This way the search would be:
index=myindex is_bad=true
as the lookup will be appended automatically
I typically use "lookup" instead of inputlookup along with a boolean column:
## baddomains.csv domain,is_bad bad.domain.com,true
index=myindex | lookup domains.csv domain OUTPUT is_bad | search is_bad=true
How about:
index=myindex [ inputlookup domains.csv | rename baddomain as domain | fields baddomain]
This should be equivalent to:
index=myindex (domain=badguy.com OR domain=donkey.com ...)
So I've become a liar. 2 issues.
Did I ever tell you you're my hero?
You're everything, everything I wish I could be.
Thanks dude, works swell!