Splunk Search

Compare multi value field with an external file

visa87
Explorer

I have a multi value field that looks similar to below
[a,b]
[a,b,c]
[b,c]
.... so on

For each event , I want to compare this with an external file that maintains a list and my final result should be the values that do not match

For eg : my external file contains the following values
a b c d e f

My final result should look like sonething below :
[c,d,e,f]
[d,e,f]
[a,d,e,f]

Can this be achieved in Splunk ?

Tags (1)

sideview
SplunkTrust
SplunkTrust

I'm assuming your external file is a Splunk lookup, or that you can make it a Splunk lookup.
And that your [a,b,c,d,e,f] field is called "foo", and say there's at least one other field in that lookup that is present on all rows, that we'll call "bar".
And also let's assume that in the raw data your multivalue field is called mvField.

<your searchterms> 
| fields mvField (and whatever few other fields you will ultimately need)
| streamstats count as rowId 
| mvexpand mvField 
| lookup foo as mvField OUTPUT bar 
| where isnull(bar)
| stats values(*) as * by rowId

The idea is to use streamstats to paint a unique id on each row, mvexpand out your multivalue field, wash it through a lookup, then discard the rows that actually matched the lookup. Then finally we use stats values(*) to fold all the expanded rows back up, and end up with mvField being a multivalue field again.

0 Karma
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...