Splunk Search

Compare multi value field with an external file

visa87
Explorer

I have a multi value field that looks similar to below
[a,b]
[a,b,c]
[b,c]
.... so on

For each event , I want to compare this with an external file that maintains a list and my final result should be the values that do not match

For eg : my external file contains the following values
a b c d e f

My final result should look like sonething below :
[c,d,e,f]
[d,e,f]
[a,d,e,f]

Can this be achieved in Splunk ?

Tags (1)

sideview
SplunkTrust
SplunkTrust

I'm assuming your external file is a Splunk lookup, or that you can make it a Splunk lookup.
And that your [a,b,c,d,e,f] field is called "foo", and say there's at least one other field in that lookup that is present on all rows, that we'll call "bar".
And also let's assume that in the raw data your multivalue field is called mvField.

<your searchterms> 
| fields mvField (and whatever few other fields you will ultimately need)
| streamstats count as rowId 
| mvexpand mvField 
| lookup foo as mvField OUTPUT bar 
| where isnull(bar)
| stats values(*) as * by rowId

The idea is to use streamstats to paint a unique id on each row, mvexpand out your multivalue field, wash it through a lookup, then discard the rows that actually matched the lookup. Then finally we use stats values(*) to fold all the expanded rows back up, and end up with mvField being a multivalue field again.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...