topic Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results? in Splunk Search

Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

driva — Wed, 29 Jan 2020 16:17:24 GMT

Apologies if the title of the question is a bit vague!

I have search that is creating a table based on events that contain a word in a lookup CSV file. This works well, however I'm trying to prevent 'words within words' appearing in the output. For example, if my lookup file contains the word 'kill', I do no want to see the word 'skills' in my results. The field name in the CSV is 'HighRiskWords'.

Here's what Im working with so far:

index=web_filter  

    [| inputlookup highriskwords.csv  

    | eval HighRiskWords="*"+HighRiskWords+"*"  

    | rename HighRiskWords as web_HighRisk]

If I use: eval HighRiskWords=HighRiskWords I get results that offer an exact match. If I use eval HighRiskWords=""+HighRiskWords+"*"* I get all matches plus any other text string surrounding the matching word, e.g: skills.

It would be wonderful to put a space in at the end of the eval command like: eval HighRiskWords=HighRiskWords+" " however this does not work.

Would anyone be able to show me how to add a space to the end of the lookup field so that I do not get 'word within words'. I want to see results like: 'biggest kill' or 'kill time', not 'top 10 skills'.

Hopefully that makes sense! Thanks for your help!

Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

gcusello — Wed, 29 Jan 2020 16:22:38 GMT

Hi @driva,
I'm not sure to have understood your need: you want to search in a log using the values in a field of a lookup as full text search, is it correct?

If this is your need, you could try something like this

index=web_filter  [ | inputlookup highriskwords.csv  | rename HighRiskWords AS query | fields query ]  
| ...

Ciao.
Giuseppe

Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

driva — Wed, 29 Jan 2020 16:26:09 GMT

Hi Giuseppe, sorry no, I haven't made myself clear... I believe the focus here is on the eval command. Is it possible to change the field so that it includes a literal space at the end of it? For example: eval HighRiskWords=HighRiskWords+" " <-- Space?

Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

gcusello — Wed, 29 Jan 2020 16:39:23 GMT

Hi @driva,
ok, sorry for the misunderstand.
yes you can, you have to use dot instead +:

index=web_filter  
      [| inputlookup highriskwords.csv  
       | eval HighRiskWords="*".HighRiskWords."*"  
       | rename HighRiskWords AS web_HighRisk
      ]
| ...

or adding a space | eval HighRiskWords=HighRiskWords." ".

Ciao.
Giuseppe

Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

driva — Wed, 29 Jan 2020 17:00:58 GMT

Hi Giuseppe,

Thanks for your reply, unfortunately . and + behave the same way? I'm still seeing words like 'skills' appear when using the .

Kind regards,
D

Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

driva — Wed, 29 Jan 2020 17:03:41 GMT

@gcusello Here's the full search:
index=web_filter

        [| inputlookup highriskwords.csv  

        | eval HighRiskWords="*".HighRiskWords."*"  

        | rename HighRiskWords as web_HighRisk]  

    | stats count by web_HighRisk, web_User, _time  

    | rex field=web_HighRisk max_match=10  

        [| inputlookup highriskwords.csv  

        | table HighRiskWords  

        | stats values(HighRiskWords) AS HighRiskWords  

        | eval search="\"(?<Matched_Word>(".mvjoin(HighRiskWords,"|")."))\""  

        | fields search]  

    | table Matched_Word, web_HighRisk, web_User, _time  

    | sort Matched_Word

Re: Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

gcusello — Wed, 29 Jan 2020 17:11:17 GMT

Hi @driva,
edit the transforms.conf where your lookup is defined and add to its stanza match_type = WILDCARD, restart splunk and try again.
For more infos see at https://docs.splunk.com/Documentation/Splunk/8.0.1/Admin/Transformsconf .

Ciao.
Giuseppe