topic Re: Look up table question in Splunk Search

Look up table question

xvxt006 — Thu, 19 Sep 2013 22:54:49 GMT

Hi,

We have a filed called BOTs which extracts all the legitimate BOTs (which have +http://.... in the user agent). I want to add the other BOTs into the same field which does not follow the standard user agent format (so they won't have +http://.. format).

I have a look up table and tried the match_type = WILDCARD(useragent) and then i have in the csv file (Look up file).
BOTs useragent
Traverse Traverse
Capture Capture
But i am not getting in the BOTs field. Any suggestions?

Re: Look up table question

the_wolverine — Thu, 19 Sep 2013 23:37:42 GMT

Assuming your props/transforms is properly configured, your csv file fields will need to include * (one or both sides of the value) in order to wildcard match.

Re: Look up table question

xvxt006 — Fri, 20 Sep 2013 02:04:00 GMT

I forgot to put asterisk in here but I do have them in my cvs file. But still not seeing them.

Re: Look up table question

lukejadamec — Mon, 28 Sep 2020 14:49:09 GMT

You might need to add:

case_sensitive_match = false

to the transforms.conf stanza for this input.

The problem with this attribute is it needs to be reset after every splunk_web lookup update/save.

Re: Look up table question

xvxt006 — Fri, 20 Sep 2013 03:21:21 GMT

Thank you. I will try this. Can you elaborate on "it needs to be reset after every splunk_web lookup update/save"

Re: Look up table question

xvxt006 — Mon, 28 Sep 2020 14:50:12 GMT

Hi, i have added that but still i don't see data

[BOTs]
filename = BOTs.csv
case_sensitive_match=false
match_type = WILDCARD(User_Agent)

Re: Look up table question

the_wolverine — Tue, 24 Sep 2013 20:08:29 GMT

Do you have a sample of your current lookup file? It needs to be a comma delimited csv file with wildcard (*).

To see the output, type in search UI, " | inputlookup BOTs.csv "

Also, make sure that the lookup file exists in ~/etc/system/lookups/ OR ~/etc/apps/search/lookups/ and has read permission properly set.

Re: Look up table question

lukejadamec — Mon, 28 Sep 2020 14:50:15 GMT

From Manager>Lookups>Lookup Definitions or Automatic Lookups for this lookup, if you open it, and save it, the case_sensitive_property will go away in transforms.conf.
I have found a way to make sure the case_sensitive_match=false is not reset. In transforms.conf add the stanza to the [default] level. Howver, this will effect all lookups in that transforms.conf