Splunk Search

Input Lookup: How can I Edit a Lookup Field with 'eval' command or 'RegEx' to narrow down my search results?

driva
Path Finder

Apologies if the title of the question is a bit vague!

I have search that is creating a table based on events that contain a word in a lookup CSV file. This works well, however I'm trying to prevent 'words within words' appearing in the output. For example, if my lookup file contains the word 'kill', I do no want to see the word 'skills' in my results. The field name in the CSV is 'HighRiskWords'.

Here's what Im working with so far:

index=web_filter  

    [| inputlookup highriskwords.csv  

    | eval HighRiskWords="*"+HighRiskWords+"*"  

    | rename HighRiskWords as web_HighRisk]  

If I use: eval HighRiskWords=HighRiskWords I get results that offer an exact match. If I use eval HighRiskWords=""+HighRiskWords+"*"* I get all matches plus any other text string surrounding the matching word, e.g: skills.

It would be wonderful to put a space in at the end of the eval command like: eval HighRiskWords=HighRiskWords+" " however this does not work.

Would anyone be able to show me how to add a space to the end of the lookup field so that I do not get 'word within words'. I want to see results like: 'biggest kill' or 'kill time', not 'top 10 skills'.

Hopefully that makes sense! Thanks for your help!

0 Karma

gcusello
Legend

Hi @driva,
I'm not sure to have understood your need: you want to search in a log using the values in a field of a lookup as full text search, is it correct?

If this is your need, you could try something like this

index=web_filter  [ | inputlookup highriskwords.csv  | rename HighRiskWords AS query | fields query ]  
| ...

Ciao.
Giuseppe

0 Karma

driva
Path Finder

Hi Giuseppe, sorry no, I haven't made myself clear... I believe the focus here is on the eval command. Is it possible to change the field so that it includes a literal space at the end of it? For example: eval HighRiskWords=HighRiskWords+" " <-- Space?

0 Karma

gcusello
Legend

Hi @driva,
ok, sorry for the misunderstand.
yes you can, you have to use dot instead +:

index=web_filter  
      [| inputlookup highriskwords.csv  
       | eval HighRiskWords="*".HighRiskWords."*"  
       | rename HighRiskWords AS web_HighRisk
      ]
| ...

or adding a space | eval HighRiskWords=HighRiskWords." ".

Ciao.
Giuseppe

0 Karma

driva
Path Finder

Hi Giuseppe,

Thanks for your reply, unfortunately . and + behave the same way? I'm still seeing words like 'skills' appear when using the .

Kind regards,
D

0 Karma

gcusello
Legend

Hi @driva,
edit the transforms.conf where your lookup is defined and add to its stanza match_type = WILDCARD, restart splunk and try again.
For more infos see at https://docs.splunk.com/Documentation/Splunk/8.0.1/Admin/Transformsconf .

Ciao.
Giuseppe

0 Karma

driva
Path Finder

@gcusello Here's the full search:
index=web_filter

        [| inputlookup highriskwords.csv  

        | eval HighRiskWords="*".HighRiskWords."*"  

        | rename HighRiskWords as web_HighRisk]  

    | stats count by web_HighRisk, web_User, _time  

    | rex field=web_HighRisk max_match=10  

        [| inputlookup highriskwords.csv  

        | table HighRiskWords  

        | stats values(HighRiskWords) AS HighRiskWords  

        | eval search="\"(?<Matched_Word>(".mvjoin(HighRiskWords,"|")."))\""  

        | fields search]  

    | table Matched_Word, web_HighRisk, web_User, _time  

    | sort Matched_Word  
0 Karma
Get Updates on the Splunk Community!

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Want a chance to win $500 to the Splunk shop? Take our IT Incident Management Survey!

  Top Trends & Best Practices in Incident ManagementSplunk is partnering up with Constellation Research to ...