Utilizing dynamic filter with inputlookup subsearch

[harry1]

Hi,

I am having a situation where a lookup table defines search filters that needs to be used as part of search query. The dynamic filter (data_owner_filter) is built from original search results and subsearch filters are defined by lookup table, where filters can either be inclusive or exclusive.

I have tried with a following kind of approach, but the problem of subsearch not being able to reach value defined as data_owner_filter:

<search>
| eval data_owner_filter=mvindex(split(data_owner,"_"),1)
| search ([| inputlookup lookup_table.csv | search static_filter="use_case_1" dynamic_filter=data_owner_filter rule_type="inclusive" | fields fieldx])
| search NOT ([| inputlookup lookup_table.csv | search static_filter="use_case_1" dynamic_filter=data_owner_filter rule_type="exclusive" | fields fieldx])
| table fieldx, fieldy, data_owner

 
Example of the lookup table (table can have hundreds of entries):
static_filter  | dynamic_filter | rule_type | fieldx
use_case_1 | 001                       | inclusive  | abc*
use_case_1 | 001                       | exclusive | efg*
use_case_1 | 002                       | inclusive  | bcd*
use_case_1 | 002                       | inclusive  | abc*
use_case_2 | 002                       | inclusive  | abc*
use_case_2 | 002                       | exclusive | hij*
...

The idea behind the whole approach is to have a single lookup table to handle various inclusions and exclusions for data related to different data owners (owner defined on data_owner_filter) while having a single search alert configured per use case (defined by "static_filter").

Any suggestion how this could be accomplished?

[to4kawa · Accepted solution]

<search>
| eval data_owner_filter=mvindex(split(data_owner,"_"),1)
| lookup lookup_table.csv dynamic_filter as data_owner_filter fieldx OUTPUTNEW 
| table fieldx, fieldy, data_owner ,rule_type, static_filter
| search as_you_want

e.g. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Lookup

[harry1]

Thanks. Figured out that it would be doable the way you mentioned, but since the amount of fields from lookup table is changing once in a while and the resulting search being hard to maintain, I decided it is better to split the search into two searches and do the dynamic part of filtering on the second search.