I've found some variations on this issue but nothing exactly the same. Go easy on me...
I'm dealing with events that includes a space delimited field containing column names. Over the years, we've pinpointed columns that we've learned do not include any data we need to be concerned about and have built a lookup table with the specific column names.
As the query is currently built, we are excluding any event that matches a value in the lookup table, regardless of what other column name is present. The problem is we want to see events that include columns not found in the lookup. For example:
bad_column1 bad_column37 (excluded, both bad columns)
bad_column column_to_investigate (should not be excluded)
I've been toying with makemv to try and parse out the columns on an individual basis but I've only been able to get the basic implementation to work where any event with a bad column is excluded (I wrote the below from memory so it may have some syntax errors).
base search NOT [ | inputlookup "bad_columns.csv" | rename bad_columns as
query ] | table field1, field2, Columns
I'm not married to the multivalue solution but it seems like there's something there. If there's a better suggestion, by all means let me know.
Edit: I have a query that working somewhat but I lose precision during large scans (event counts are lower than the actual number of events). Any assistance is appreciated:
base search | eval UID = _cd | eval singleColumns=split(column_name, " ") | mvexpand
singleColumns | search NOT [|inputlookup Known_Bad_Columns | rename bad_columns as
singleColumns ] | dedup UID | stats count by field1, field2 | sort by count desc