topic Re: Remove field values from one multi-valued field which values are present in another multi-valued field in Splunk Search

Remove field values from one multi-valued field which values are present in another multi-valued field

VatsalJagani — Tue, 05 Jul 2022 11:44:36 GMT

Remove field values from one multi-valued field which values are present in another multi-valued field

Looking for something like:

| eval dest=mvfilter(if(dest IN email_sender, null(), dest))

Here dest contains both sender and receiver of the email. hence I'm trying to exclude the sender from it.

(FYI, the sender is also a multi-valued field that's because I've used stats before it.)

Re: Remove field values from one multi-valued field which values are present in another multi-valued field

bowesmana — Wed, 06 Jul 2022 02:20:44 GMT

Use mvmap. See this example

last line removes 'User3' from the dest field as it's one of the senders.

Re: Remove field values from one multi-valued field which values are present in another multi-valued field

yuanliu — Wed, 06 Jul 2022 02:25:40 GMT

If I'm not mistaken, mvfilter() does just that. You can use mvmap() to iterate over email_sender.

| eval dest=mvmap(email_sender, mvfilter(isnull(mvfind(dest, "^" . email_sender . "$"))))

The mvfind() expression assumes that each email_sender would match the exact spelling if it appears in dest.

Re: Remove field values from one multi-valued field which values are present in another multi-valued field

VatsalJagani — Wed, 06 Jul 2022 06:37:57 GMT

Thanks @yuanliu. Logically I thought this should work but somehow mvfilter doesn't want to work. Anyways it worked with @bowesmana answer without mvfilter.

Re: Remove field values from one multi-valued field which values are present in another multi-valued field

yuanliu — Thu, 07 Jul 2022 02:55:29 GMT

I think I know why mvfilter gives error. mvfilter requires its argument to only involve one multivalue variable. But because of mvfind, it now involves both dest and email_sender, even though email_sender is actually single-valued inside the mvmap iterator. In fact, mvfilter will parse to error even if email_sender is genuinely single valued.

There might be some roundabout way to turn email_sender into a pattern substitution instead of a variable, but that is in itself too convoluted.

Re: Remove field values from one multi-valued field which values are present in another multi-valued field

VatsalJagani — Thu, 07 Jul 2022 04:56:57 GMT

@yuanliu - Yeah, mvfilter can reference only one field, the rest should be only string/pattens.

The expression can reference only one field.

(From doc - https://docs.splunk.com/Documentation/SCS/current/SearchReference/MultivalueEvalFunctions)