How to set up a blacklist lookup to run searches against and alert if there are any matching field values?

[j666gak]

Hello,

I needed a blacklist to populate with IPs and URLs, so I created a lookup file called blacklist.csv. the Lookup file had two fields defined called id and uri. I have populate a couple of URLs that I know are in the index holding data from the webproxy in a field extract called url. all I want is to create an alert so when something appears in the the URL field that also appears in the uri field of the blacklist lookup.

I have tried running the search below, but it just returns all logs and not just the URLs from the lookup

index=bloxx |lookup blacklist.csv uri

I would really appreciate any help.

Thank

[j666gak · Accepted solution]

I managed to have a copy of the lookup within the app and my user profile, which caused the issue.

[martin_mueller]

Run this instead:

index=bloxx [inputlookup blacklist.csv | fields uri]

That'll create a long OR'd filter from your CSV file and use that to search the data.

[martin_mueller]

Check the job inspector, it should tell you what the subsearch returned. See if that search string makes sense or not.

[j666gak]

I managed to have a copy of the lookup within the app and my user profile, which caused the issue. Thank you for your help though.

[j666gak]

No, unfortunately not. No it just says there are no result after it has searched through them all. I know there is data it should fire on, it is really frustrating me.

I have checked permission on the index/lookup/role/NTFS, removed and re-added the lookup and lookup definition, can see entries being searched from the lookup file.

Don't know what I am doing or is going wrong?

[martin_mueller]

So... it's working fine now?

[martin_mueller]

Does your data have a uri field extracted? If not, extract that field and you'll be good.

[j666gak]

I changed the field within the lookup file from uri to url, and there is already a field extract within the Bloxx source for url.

[martin_mueller]

Check the job inspector for what's being returned by the subsearch and see if that makes sense for your data. It should look something like this:

((uri="...") OR (uri="...") ...)

Then make sure your data actually has a field called uri with such values.

[j666gak]

Yes I can see that, and they show items from the correct field in the lookup file.

If I run the command within job manually, I have to remove the uri= for it to work, how do I do this automatically?

[j666gak]

Many thanks for the response.

I am trying the command but it is not returning any results, I have confirmed the time period and that URL that has been accessed is in the lookuptable.

I think some kind of wildcard may be needed?