Splunk Search

grep -f over multiple fields

murhammr
Path Finder

i'm trying to do something similar to grep -f over multiple sourcetypes that i've appended together into one search.

Example: index=xyz sourcetype=watchlist | fields name
looks like this

zeng
smith
willi
johns
charl

the sourcetypes i want to search may contain any of these fields: username, lastname, fullname
and i want to find all the name values in the watchlist in any of username or lastname or fullname or any other field in those events

i thought subsearches would work but the field name in the watchlist differ from the multiple field names in the other sourcetypes, and it's not an = search, it's a wildcard match search.

0 Karma
1 Solution

micahkemp
Champion

If you know that the value you're looking for is already extracted into a field, and you need to verify that the value exists in a field (but you don't want to define a list of all possible fields), something like this run anywhere search should work for you:

| makeresults | eval firstname="personA"
| append [| makeresults | eval lastname="personA"]
| append [| makeresults | eval firstname="personB"]
| append [| makeresults | eval anyfieldname="personA"]
| foreach * [ eval found=if(<<FIELD>>="personA", "YES", found)]
| search found="YES"

The logic is all in the last two lines, making use of the foreach command to iterate through all fields to find a match. In my example I have it looking for the value personA in any field.

View solution in original post

0 Karma

micahkemp
Champion

If you know that the value you're looking for is already extracted into a field, and you need to verify that the value exists in a field (but you don't want to define a list of all possible fields), something like this run anywhere search should work for you:

| makeresults | eval firstname="personA"
| append [| makeresults | eval lastname="personA"]
| append [| makeresults | eval firstname="personB"]
| append [| makeresults | eval anyfieldname="personA"]
| foreach * [ eval found=if(<<FIELD>>="personA", "YES", found)]
| search found="YES"

The logic is all in the last two lines, making use of the foreach command to iterate through all fields to find a match. In my example I have it looking for the value personA in any field.

0 Karma

murhammr
Path Finder

thanks! this was the basis for my solution. the multivalue field result will contain a yes if any of that event's fields is a match.

| makeresults | eval name5="son" | eval firstname="personA" 
| append     [| makeresults     | eval name5="sonA"     | eval lastname="personA"] 
| append     [| makeresults     | eval name5="sonA"     | eval firstname="personB"] 
| append     [| makeresults     | eval name5="per"     | eval anyfieldname="personA"] 
| eval result="" 
| foreach *name 
    [ eval result=if(like('<<FIELD>>',"%".name5."%"),result + " yes", result + " no") ] 
| makemv   result |search result=yes
0 Karma

493669
Super Champion

If I understood correctly you have multiple sourcetypes ...if you have one index pointing multiple sourcetypes then you can try like:

index=<indexname>

and if you have different sourcetype pointing different index then try like this

index=<indexname1> sourcetype=<sourcetype_name1> OR (index=<indexname2> sourcetype=<sourcetype_name2>)

and if you want to find particular name (abc) contain any of these fields: username, lastname, fullname then you can try like function

...|eval matchTest=case((like(username, "%abc%")), "matched",(like(lastname, "%abc%")),"matched",(like(fullname, "%abc%")),"matched",1=1,"notMatched")
0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...