Is it possible to remove values specified in a field from a list of values in another field?

[andrewtrobec]

Hello,

Given two list fields add and remove, as well as a currentList list field, is it possible to create a new fieldupdatedListso that gets updated with add values and without remove values?

add,remove,currentList,updatedList
value1,value2,value2 value3 value4, value1 value3 value4
value5,value3 value4,value1 value3 value4,value1 value5

Any hints would be great!

Best regards,

Andrew

[micahkemp · Accepted solution]

Here's a run-anywhere possibility:

| makeresults | eval add="value1", remove="value2", current="value2 value3 value4"
| append [| makeresults | eval add="value5", remove="value3 value4", current="value1 value3 value4"]
| makemv add 
| makemv remove 
| makemv current
| eval current_plus_add=mvappend(add, current)
| streamstats window=1 current=t values(current_plus_add) AS distinct_current_plus_add, values(remove) AS distinct_remove
| eval distinct_current_plus_add_plus_distinct_remove=mvappend(distinct_current_plus_add, distinct_remove)
| streamstats count AS serial
| stats count BY distinct_current_plus_add_plus_distinct_remove serial
| search count=1
| stats list(distinct_current_plus_add_plus_distinct_remove) AS after_add_remove BY serial

[andrewtrobec]

Beautiful! I hope that a future feature for multivalues will be mvremove or something like that so we can easily remove values instead of tricky workarounds!

[niketn]

@andrewtrobec, you can create your own custom commands 🙂

https://docs.splunk.com/Documentation/Splunk/latest/Search/Customsearchcommandshape
http://dev.splunk.com/view/python-sdk/SP-CAAAEU2

[somesoni2]

Is field currentList multivalued field? Also, are add and remove multivalued field or single valued?

[andrewtrobec]

currentList, add, and remove are all multivalue fields. I've figured the add part out:

| eval currentList=mvdedup(mvappend(currentList, add))

I just need to figure out the remove part.

[somesoni2]

Unfortunately, there is no easy/direct way for removal from multivalued field. One method could be like this.

your current search with fields add, remove and currentList
| eval currentList=mvdedup(mvappend(currentList, add))
| nomv remove 
| streamstats count as sno
| mvexpand currentList
| where len(replace(remove,currentList,"")=len(remove)
| stats values(*) as * by sno | fields - sno

[andrewtrobec]

I've cut and pasted the code into my search but it doesn't seem to work, it just kinda messes up the fields. I'll deconstruct the logic and play around with it. Thanks!