topic Re: How to use inputlookup to filter in Getting Data In

How to use inputlookup to filter

Hung_Nguyen — Tue, 29 Sep 2020 08:54:04 GMT

Hi,

I have multiple queries that I use to do daily report on errors in our production Splunk. I would like to filter out known issues so the report is less cluttered with known issues. I have create a lookup file, let's say "foo.csv", which has content:

known_issues_strings
NOT "known string"
NOT "known issue1"
NOT "known issue2"
NOT "known issue3"
etc .....

Currently my search is like this:

source=*logger* NOT "known string" NOT "known issue1"  NOT "known issue2" NOT "known issue3"

How do I use inputlookup so that I don't need to spell out all the filtering strings in each of my report searches? thanks

Re: How to use inputlookup to filter

stephanefotso — Tue, 29 Sep 2020 08:54:07 GMT

Hello. Search the foo.csv lookup file (under $SPLUNK_HOME/etc/system/lookups or $SPLUNK_HOME/etc/apps//lookups). like this:

| inputlookup foo.csv

For more informations about the inputlookup command read this: http://docs.splunk.com/Documentation/Splunk/6.3.3/SearchReference/Inputlookup
Thanks

Re: How to use inputlookup to filter

HeinzWaescher — Wed, 24 Feb 2016 10:50:18 GMT

sourcetype=logger AND NOT [|inputlookup foo.csv | fields+ known_issue_strings | rename known_issue_strings AS "your_error_field"]

Re: How to use inputlookup to filter

fdi01 — Wed, 24 Feb 2016 11:32:59 GMT

try like :

sourcetype=logger   [|inputlookup foo.csv ] |...

Re: How to use inputlookup to filter

somesoni2 — Wed, 24 Feb 2016 15:31:03 GMT

Assuming you want to do a text search of known errors, here is what I would suggest

a) Update your lookup to just have the known error string.
foo.csv

known_issues_strings
"known string"
"known issue1"
"known issue2"
"known issue3"

Update#1

b) Update your base search like this

source=*logger* NOT [| intputlookup foo.csv | eval search="\".known_issues_strings."\"" | table search  ]

Re: How to use inputlookup to filter

dcarmack_splunk — Wed, 24 Feb 2016 19:24:28 GMT

I think this list would be easier to maintain in a macro, which is simply a condensed search string held in a Splunk knowledge object.

http://docs.splunk.com/Documentation/Splunk/6.0.4/Search/Usesearchmacros

If you insist on a lookup table and intend to search the values as raw strings in the events, you will need to rename the lookup table header field to "query". Query is a reserved field name that allows this type of behavior.

| inputlookup foo.csv | rename myfield AS query | fields query

Re: How to use inputlookup to filter

Hung_Nguyen — Wed, 24 Feb 2016 20:15:15 GMT

With your suggestion, this is the produced query

index=tto* NOT ( ( known_issues="known issue1" ) OR ( known_issues="known issue1" ) )

so it doesn't produce the right result. is it possible for me to tell splunk not the use the field? one way I can think of is change the csv column header to be the same as my field name and add wildcard

my_field_name
 "*known issue1*"
 "*known issue2*"

so it would produce

index=tto* NOT ( ( my_field_name="*known issue1*" ) OR ( my_field_name="*known issue1*" ) )

Although i wonder if the wild card can have a perf hit.

Re: How to use inputlookup to filter

somesoni2 — Wed, 24 Feb 2016 20:24:20 GMT

Try the updated search, which will append double quotes around the values it retrieved from lookup.