Splunk Search

Compare multi value field with an external file

visa87
Explorer

I have a multi value field that looks similar to below
[a,b]
[a,b,c]
[b,c]
.... so on

For each event , I want to compare this with an external file that maintains a list and my final result should be the values that do not match

For eg : my external file contains the following values
a b c d e f

My final result should look like sonething below :
[c,d,e,f]
[d,e,f]
[a,d,e,f]

Can this be achieved in Splunk ?

Tags (1)

sideview
SplunkTrust
SplunkTrust

I'm assuming your external file is a Splunk lookup, or that you can make it a Splunk lookup.
And that your [a,b,c,d,e,f] field is called "foo", and say there's at least one other field in that lookup that is present on all rows, that we'll call "bar".
And also let's assume that in the raw data your multivalue field is called mvField.

<your searchterms> 
| fields mvField (and whatever few other fields you will ultimately need)
| streamstats count as rowId 
| mvexpand mvField 
| lookup foo as mvField OUTPUT bar 
| where isnull(bar)
| stats values(*) as * by rowId

The idea is to use streamstats to paint a unique id on each row, mvexpand out your multivalue field, wash it through a lookup, then discard the rows that actually matched the lookup. Then finally we use stats values(*) to fold all the expanded rows back up, and end up with mvField being a multivalue field again.

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...