How can i search in logs for mutiple vaules from a file

[ma_anand1984]

I want to search logs for a list of keywords i have, say i have hundred strings. How can i achieve that in splunk?

[MHibbin · Accepted solution]

Porbably you best bet, is to have a lookup file containing you list of strings, which will make it easier to maintain (rather than multiple searches with lists) of strings)

You could then use the inputlookup command in a sub-search, to generate your list of words. Then apply this to your base level search command. This would use something like the following, as the search string (source can substituted with sourcetype, host, *, etc):

source=<someSource> [|inputlookup <someLookup> | fields <field> | rename <field> as search]

For example (not enitrely tested), on some SNMP logs, looking for certain severity keywords:

source=snmp [|inputlookup severity | fields sev | rename sev as search]

When the SNMP messages are contained in the "snmp" source, "severity" is the reference for the lookup in transforms.conf, and "sev" is the field containing my list of key severity terms.

Some references:

http://docs.splunk.com/Documentation/Splunk/5.0/Knowledge/Addfieldsfromexternaldatasources
http://docs.splunk.com/Documentation/Splunk/5.0/Search/Aboutsubsearches
http://docs.splunk.com/Documentation/Splunk/5.0/SearchReference/Inputlookup
http://docs.splunk.com/Documentation/Splunk/5.0/SearchReference/Fields

[ma_anand1984]

trying hard for more than a week. seems my creativity is not good enough for this scenario :(. Any particular sub search commands that comes to your mind?

[MHibbin]

I'm not too sure, and currently have limited access to a Splunk install. But you should definately look at some of the other sub search commands, I think there will be a solution to help, you will probably need to run another sub-search to generate those results.

[ma_anand1984]

@MHibbin
Im also interested in getting count of the matched string or atleast just list of matched string. is it possible?

[MHibbin]

... glad it helped anyway

[MHibbin]

You can use the format command outside of the subsearch (i.e. contained in the []) to check the format of the new "search" field... for example:

|inputlookup severity | fields sev | rename sev as search | format

Bit odd that you still have to use that in the subsearch

[ma_anand1984]

@MHibbin
I have asked a follow-up question here, let me know if you have ideas 🙂

http://splunk-base.splunk.com/answers/61146/search-mutiple-strings-in-logs-and-give-count-of-respect...

[ma_anand1984]

Yeah.. For some reason i have to explicitly give | format at the end in the sub search. it worked with the below command.

source=snmp [|inputlookup severity | fields sev | rename sev as search | format]

Thanks again

[MHibbin]

did you see the edits to my searches above?

[MHibbin]

@ma_anand1984, please see updated search strings... I forgot to rename the field as "search" which passes it as a search string.

[MHibbin]

...

source=snmp field1=foo OR field1=bar OR field1=blah

You may just need to test some different strings out to see what works best for you.

[MHibbin]

That search above, should search in the raw event for any of the keywords that delivered from the lookup files sev field. For example, if the severity lookup contains the words: foo; bar; blah. Then search would equate to:

source=snmp foo OR bar OR blah

I believe you can append this subsearch to a specific field if you already have a field extracted in you logs, so you could then search for strings in a specific field (e.g. field1), like:

source=snmp field1=[|inputlookup severity | fields sev |rename sev as search]

Which I think would be equivalent to:

....