Splunk Search

Compare multi value field with an external file

visa87
Explorer

I have a multi value field that looks similar to below
[a,b]
[a,b,c]
[b,c]
.... so on

For each event , I want to compare this with an external file that maintains a list and my final result should be the values that do not match

For eg : my external file contains the following values
a b c d e f

My final result should look like sonething below :
[c,d,e,f]
[d,e,f]
[a,d,e,f]

Can this be achieved in Splunk ?

Tags (1)

sideview
SplunkTrust
SplunkTrust

I'm assuming your external file is a Splunk lookup, or that you can make it a Splunk lookup.
And that your [a,b,c,d,e,f] field is called "foo", and say there's at least one other field in that lookup that is present on all rows, that we'll call "bar".
And also let's assume that in the raw data your multivalue field is called mvField.

<your searchterms> 
| fields mvField (and whatever few other fields you will ultimately need)
| streamstats count as rowId 
| mvexpand mvField 
| lookup foo as mvField OUTPUT bar 
| where isnull(bar)
| stats values(*) as * by rowId

The idea is to use streamstats to paint a unique id on each row, mvexpand out your multivalue field, wash it through a lookup, then discard the rows that actually matched the lookup. Then finally we use stats values(*) to fold all the expanded rows back up, and end up with mvField being a multivalue field again.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...