topic Re: inputlookup and substring search in Splunk Search

inputlookup and substring search

igorbukanov — Wed, 08 Aug 2012 08:10:47 GMT

We need to search if a part of URL in the log matches a string from an external file. As I can see, the method from http://splunk-base.splunk.com/answers/2457/inputlookup-against-a-list-of-bad-domains does not work as it matches against whole field values and we need to check for a substring match.

Currently as a workaround we use the command-line search and assemble the search in a shell script from the file so it looks like

search ... "*string1*" OR "*string2*" OR "*string3*" ...

But perhaps there is a better/faster way of doing the search especially given that our search list has been growing.

ziegfried — Wed, 08 Aug 2012 08:19:45 GMT

You can use a subsearch to create the wildcard-query from a lookup file (field1 is the csv column that contains the fragment to look for)

... [ | inputlookup mylookup | eval search="*"+field1+"*" | fields search ]

will expand to

... (( *value1* ) OR ( *value2*) ...

The values might need to be quoted:

... [ | inputlookup mylookup | eval search="\"*"+replace(field1,"\"","\\\"")+"*\"" | fields search ]

will expand to

... ( ( "*value1*" ) OR ( "*value2*" ) OR ( "*some value3*" ) OR ...

ziegfried — Wed, 08 Aug 2012 12:01:24 GMT

If that solves your problem, please accept the answer by clicking on the checkmark on the left.