I have some customer provided CSV lookup files. These lookup files have some "similar" field names, which means they contain some common keywords. I would like do a keyword match in lookup command to these similar fields. Is there a way to do this ?
An example is:
lookup file1's title is like: population, average income, location
lookup file2's title is like: population, income, location
I would like to use the field that contains "income" as the lookup condition, how can I do this with one common lookup statement?
You are misunderstanding the 2 different ways to use lookup files. One way is with the ... | lookup
command syntax, which uses the WILDCARD()
syntax (among other settings) within the Lookup definitions
, the other is with the |inputlookup
command syntax which DOES NOT interact with the Lookup definitions
. In the latter case, just do something like this:
index=_internal [ |inputlookup hosts_reporting.csv | eval host=host + "*" ]
Very interesting @woodcock - thank you for the clarification.
If you have your answer, pick one and click Accept
to close the question.
Hi @woodcock - it's not mine - I just joined the ride ; -)
See here for how to lookup with RegEx:
https://answers.splunk.com/answers/386488/regex-in-lookuptable.html
According to How to use wildcard in lookup-based searches and alerts?
You can specify -
match_type = WILDCARD(income)
In the transforms.conf
definition of your lookup.
I just tested it, I have -
$SPLUNK_HOME/etc/apps/search/local
$ cat transforms.conf
[hosts_reporting]
batch_index_query = 0
case_sensitive_match = 1
filename = hosts_reporting.csv
match_type = WILDCARD(host)
One of the host
names in hosts_reporting.csv
is the beginning of a host
name and it comes up via -
index=_internal [ | inputlookup hosts_reporting.csv | eval host=host + "*" ]
But it doesn't when running -
index=_internal [ | inputlookup hosts_reporting ]
Isn't it weird?
Any thoughts on this one, by any chance? @woodcock?
How are you planning to run lookup on both lookup table files? What all have you tried so far? Any specific reason to have common lookup statement?
Basically, I am running an app which processes a lookup file with some data files. The lookup csv file, which is generated from reporting systems of different vendors, varied a little bit in title fields, but the keywords are basically the same . Since there are a lot of customers, it would be easy to manage if I have one lookup statement to support all these lookup files. Now I am trying to make it a rule to all the customers, asking for manually editing the titles to make them the same, but it would still be desirable to tolerate some kind of fault or difference.