Remove field values from one multi-valued field which values are present in another multi-valued field

[VatsalJagani]

Remove field values from one multi-valued field which values are present in another multi-valued field

Looking for something like:

| eval dest=mvfilter(if(dest IN email_sender, null(), dest))

Here dest contains both sender and receiver of the email. hence I'm trying to exclude the sender from it.

(FYI, the sender is also a multi-valued field that's because I've used stats before it.)

[yuanliu]

If I'm not mistaken, mvfilter() does just that.  You can use mvmap() to iterate over email_sender.

| eval dest=mvmap(email_sender, mvfilter(isnull(mvfind(dest, "^" . email_sender . "$"))))

 The mvfind() expression assumes that each email_sender would match the exact spelling if it appears in dest.

[VatsalJagani]

Thanks @yuanliu. Logically I thought this should work but somehow mvfilter doesn't want to work. Anyways it worked with @bowesmana answer without mvfilter.

VatsalJagani_0-1657089389258.png

[yuanliu]

I think I know why mvfilter gives error.  mvfilter requires its argument to only involve one multivalue variable.  But because of mvfind, it now involves both dest and email_sender, even though email_sender is actually single-valued inside the mvmap iterator.  In fact, mvfilter will parse to error even if email_sender is genuinely single valued.

There might be some roundabout way to turn email_sender into a pattern substitution instead of a variable, but that is in itself too convoluted.

[VatsalJagani]

@yuanliu - Yeah, mvfilter can reference only one field, the rest should be only string/pattens.

The expression can reference only one field.

(From doc - https://docs.splunk.com/Documentation/SCS/current/SearchReference/MultivalueEvalFunctions)

[bowesmana · Accepted solution]

Use mvmap. See this example

| makeresults count=1
| eval dest=split("User1,User2,User3,User4,User5",",")
| eval sender=split("User3,User7", ",")
| table sender dest
| eval dest=mvmap(dest,if(isnull(mvfind(sender,dest)),dest,null))

last line removes 'User3' from the dest field as it's one of the senders.