topic grep -f over multiple fields in Splunk Search

grep -f over multiple fields

murhammr — Fri, 16 Feb 2018 23:09:53 GMT

i'm trying to do something similar to grep -f over multiple sourcetypes that i've appended together into one search.

Example: index=xyz sourcetype=watchlist | fields name
looks like this

zeng
smith
willi
johns
charl

the sourcetypes i want to search may contain any of these fields: username, lastname, fullname
and i want to find all the name values in the watchlist in any of username or lastname or fullname or any other field in those events

i thought subsearches would work but the field name in the watchlist differ from the multiple field names in the other sourcetypes, and it's not an = search, it's a wildcard match search.

Re: grep -f over multiple fields

493669 — Sat, 17 Feb 2018 00:33:03 GMT

If I understood correctly you have multiple sourcetypes ...if you have one index pointing multiple sourcetypes then you can try like:

index=<indexname>

and if you have different sourcetype pointing different index then try like this

index=<indexname1> sourcetype=<sourcetype_name1> OR (index=<indexname2> sourcetype=<sourcetype_name2>)

and if you want to find particular name (abc) contain any of these fields: username, lastname, fullname then you can try like function

...|eval matchTest=case((like(username, "%abc%")), "matched",(like(lastname, "%abc%")),"matched",(like(fullname, "%abc%")),"matched",1=1,"notMatched")

Re: grep -f over multiple fields

micahkemp — Sat, 17 Feb 2018 00:40:46 GMT

If you know that the value you're looking for is already extracted into a field, and you need to verify that the value exists in a field (but you don't want to define a list of all possible fields), something like this run anywhere search should work for you:

| makeresults | eval firstname="personA"
| append [| makeresults | eval lastname="personA"]
| append [| makeresults | eval firstname="personB"]
| append [| makeresults | eval anyfieldname="personA"]
| foreach * [ eval found=if(<<FIELD>>="personA", "YES", found)]
| search found="YES"

The logic is all in the last two lines, making use of the foreach command to iterate through all fields to find a match. In my example I have it looking for the value personA in any field.

Re: grep -f over multiple fields

murhammr — Mon, 19 Feb 2018 18:29:07 GMT

thanks! this was the basis for my solution. the multivalue field result will contain a yes if any of that event's fields is a match.

| makeresults | eval name5="son" | eval firstname="personA" 
| append     [| makeresults     | eval name5="sonA"     | eval lastname="personA"] 
| append     [| makeresults     | eval name5="sonA"     | eval firstname="personB"] 
| append     [| makeresults     | eval name5="per"     | eval anyfieldname="personA"] 
| eval result="" 
| foreach *name 
    [ eval result=if(like('<<FIELD>>',"%".name5."%"),result + " yes", result + " no") ] 
| makemv   result |search result=yes