We need to search if a part of URL in the log matches a string from an external file. As I can see, the method from http://splunk-base.splunk.com/answers/2457/inputlookup-against-a-list-of-bad-domains does not work as it matches against whole field values and we need to check for a substring match.
Currently as a workaround we use the command-line search and assemble the search in a shell script from the file so it looks like
search ... "*string1*" OR "*string2*" OR "*string3*" ...
But perhaps there is a better/faster way of doing the search especially given that our search list has been growing.
You can use a subsearch to create the wildcard-query from a lookup file (field1 is the csv column that contains the fragment to look for)
... [ | inputlookup mylookup | eval search="*"+field1+"*" | fields search ]
will expand to
... (( *value1* ) OR ( *value2*) ...
The values might need to be quoted:
... [ | inputlookup mylookup | eval search="\"*"+replace(field1,"\"","\\\"")+"*\"" | fields search ]
will expand to
... ( ( "*value1*" ) OR ( "*value2*" ) OR ( "*some value3*" ) OR ...
If that solves your problem, please accept the answer by clicking on the checkmark on the left.