Here's a suggestion trying to get to the same result though via a different approach and one that would not require to effectively make an all time search every hour.
To remember already discovered combinations of field1, field2, field2 long term and even exceeding your normal data retention period you could use either a summary index or a lookup table.
In general I find working with lookup tables easier so here's an suggestion facilitating one:
In CSV format the table, I named discovered_combinations, could look like:
discovered_timestamp,field1,field2,field3
1423922204,AAA,"",123
1423923303,"",bbb,435
1423924444,"RRR,bbb,""
The timestamp would provide a return value but might also be used in queries and generally help keep track of when field combinations were discovered.
A search scheduled every hour would find events that don't match a field combination in the lookup table, dedup and send the alert so you get the first event for any new combination.
index=mydata earliest=-1h@h latest=@h
| dedup field1, field2, field3
| lookup discovered_combinations field1 field2 field2 OUTPUT discovered_timestamp
| where isNull(discovered_timestamp)
A second search scheduled afterwards would then update the lookup table using ... | outputlookup discovered_combinations append=true .
There might be a way of doing this all in a single query which would be neater.
... View more