Hi,
I am having a situation where a lookup table defines search filters that needs to be used as part of search query. The dynamic filter (data_owner_filter) is built from original search results and subsearch filters are defined by lookup table, where filters can either be inclusive or exclusive.
I have tried with a following kind of approach, but the problem of subsearch not being able to reach value defined as data_owner_filter:
<search>
| eval data_owner_filter=mvindex(split(data_owner,"_"),1)
| search ([| inputlookup lookup_table.csv | search static_filter="use_case_1" dynamic_filter=data_owner_filter rule_type="inclusive" | fields fieldx])
| search NOT ([| inputlookup lookup_table.csv | search static_filter="use_case_1" dynamic_filter=data_owner_filter rule_type="exclusive" | fields fieldx])
| table fieldx, fieldy, data_owner
Example of the lookup table (table can have hundreds of entries):
static_filter | dynamic_filter | rule_type | fieldx
use_case_1 | 001 | inclusive | abc*
use_case_1 | 001 | exclusive | efg*
use_case_1 | 002 | inclusive | bcd*
use_case_1 | 002 | inclusive | abc*
use_case_2 | 002 | inclusive | abc*
use_case_2 | 002 | exclusive | hij*
...
The idea behind the whole approach is to have a single lookup table to handle various inclusions and exclusions for data related to different data owners (owner defined on data_owner_filter) while having a single search alert configured per use case (defined by "static_filter").
Any suggestion how this could be accomplished?
<search>
| eval data_owner_filter=mvindex(split(data_owner,"_"),1)
| lookup lookup_table.csv dynamic_filter as data_owner_filter fieldx OUTPUTNEW
| table fieldx, fieldy, data_owner ,rule_type, static_filter
| search as_you_want
e.g. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Lookup
<search>
| eval data_owner_filter=mvindex(split(data_owner,"_"),1)
| lookup lookup_table.csv dynamic_filter as data_owner_filter fieldx OUTPUTNEW
| table fieldx, fieldy, data_owner ,rule_type, static_filter
| search as_you_want
e.g. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Lookup
Thanks. Figured out that it would be doable the way you mentioned, but since the amount of fields from lookup table is changing once in a while and the resulting search being hard to maintain, I decided it is better to split the search into two searches and do the dynamic part of filtering on the second search.