Splunk Search

Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

Path Finder

Apologies if the title of the question is a bit vague!

I have search that is creating a table based on events that contain a word in a lookup CSV file. This works well, however I'm trying to prevent 'words within words' appearing in the output. For example, if my lookup file contains the word 'kill', I do no want to see the word 'skills' in my results. The field name in the CSV is 'HighRiskWords'.

Here's what Im working with so far:

index=web_filter  

    [| inputlookup highriskwords.csv  

    | eval HighRiskWords="*"+HighRiskWords+"*"  

    | rename HighRiskWords as web_HighRisk]  

If I use: eval HighRiskWords=HighRiskWords I get results that offer an exact match. If I use eval HighRiskWords=""+HighRiskWords+"*"* I get all matches plus any other text string surrounding the matching word, e.g: skills.

It would be wonderful to put a space in at the end of the eval command like: eval HighRiskWords=HighRiskWords+" " however this does not work.

Would anyone be able to show me how to add a space to the end of the lookup field so that I do not get 'word within words'. I want to see results like: 'biggest kill' or 'kill time', not 'top 10 skills'.

Hopefully that makes sense! Thanks for your help!

0 Karma

SplunkTrust
SplunkTrust

Hi @driva,
I'm not sure to have understood your need: you want to search in a log using the values in a field of a lookup as full text search, is it correct?

If this is your need, you could try something like this

index=web_filter  [ | inputlookup highriskwords.csv  | rename HighRiskWords AS query | fields query ]  
| ...

Ciao.
Giuseppe

0 Karma

Path Finder

Hi Giuseppe, sorry no, I haven't made myself clear... I believe the focus here is on the eval command. Is it possible to change the field so that it includes a literal space at the end of it? For example: eval HighRiskWords=HighRiskWords+" " <-- Space?

0 Karma

SplunkTrust
SplunkTrust

Hi @driva,
ok, sorry for the misunderstand.
yes you can, you have to use dot instead +:

index=web_filter  
      [| inputlookup highriskwords.csv  
       | eval HighRiskWords="*".HighRiskWords."*"  
       | rename HighRiskWords AS web_HighRisk
      ]
| ...

or adding a space | eval HighRiskWords=HighRiskWords." ".

Ciao.
Giuseppe

0 Karma

Path Finder

Hi Giuseppe,

Thanks for your reply, unfortunately . and + behave the same way? I'm still seeing words like 'skills' appear when using the .

Kind regards,
D

0 Karma

SplunkTrust
SplunkTrust

Hi @driva,
edit the transforms.conf where your lookup is defined and add to its stanza match_type = WILDCARD, restart splunk and try again.
For more infos see at https://docs.splunk.com/Documentation/Splunk/8.0.1/Admin/Transformsconf .

Ciao.
Giuseppe

0 Karma

Path Finder

@gcusello Here's the full search:
index=web_filter

        [| inputlookup highriskwords.csv  

        | eval HighRiskWords="*".HighRiskWords."*"  

        | rename HighRiskWords as web_HighRisk]  

    | stats count by web_HighRisk, web_User, _time  

    | rex field=web_HighRisk max_match=10  

        [| inputlookup highriskwords.csv  

        | table HighRiskWords  

        | stats values(HighRiskWords) AS HighRiskWords  

        | eval search="\"(?<Matched_Word>(".mvjoin(HighRiskWords,"|")."))\""  

        | fields search]  

    | table Matched_Word, web_HighRisk, web_User, _time  

    | sort Matched_Word  
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!