How to create a custom field to match a particular string?

[jwelters]

I have an interesting use case, where I have a list of strings that I search for within our proxy logs to identify problem systems. This works quite well. Currently the list exists in both a CSV format, and a search Macro. The macro just specifies (phrase1 OR phrase2 OR phrase3) it's not complicated.

Recently I decided I wanted to add to our analysis and return the results but also figure out how to add a new field that is the particular string that was matched. For example if the phrase was infected, I could display a table with the _time, URL , and matched_string. This would make analysis a bit easier for us.

I've seen a couple ways folks have looked at doing something similar but I wasn't able to find a working solution for me, has anyone done something like this before ?

[aaraneta_splunk]

@jwelters - Did one of the answers below help provide a solution your question? If yes, please click “Accept” below the best answer to resolve this post and upvote anything that was helpful. If no, please leave a comment with more feedback. Thanks.

[DalJeanis]

Here's a sample of what you can do. This makes a list (you would substitute your lookup table here)

|makeresults |eval mylist="infected disturbed reallycrazy zombiefied"|makemv mylist | mvexpand mylist 

this turns it into a tight regex

|rename mylist as search
|format "(?<resultfield>" "" 3 "" "|" ")" 
| rex field=search mode=sed "s/\>  /\>/g" 
| rex field=search mode=sed "s/  \|  /|/g"

The return value looks like this -

(?<resultfield\>infected|disturbed|reallycrazy|zombiefied )

and you can drop that into a rex command

| rex field=_raw max_match=5 [the above search using your lookup table as input]

to extract which of the lookup item(s) matched.

[somesoni2 · Accepted solution]

How many keywords are there? I'm assuming there are low as you are using them in Macro. If that's still the case, you could write another macro, which will do like this. You'd end up maintaining two macros and have to update both when you add a new phrase.

 eval matched_string=case(searchmatch("phrase1"),"phrase1",searchmatch("phrase2"),"phrase2", searchmatch("phrase3"),"phrase3",...,1=1,"PhraseNotInMacro")

[jwelters]

There are a few hundred.

[jwelters]

This was the best solution, it just wasn't a great one but it worked. I believe it's the best solution given the limitations of the core tool.

[somesoni2]

My bet will be on creating a macro with case expressions similar to above, assuming the phrase can occur at any place in the raw data (no fixed position).

[jwelters]

It looks like this will work well, as there's already a script that creates the Search macro, and the lookup table... it's fairly easy to also have it create the macro for this.