Why is there an All Time (real-time) Alert Trigger in Splunk Enterprise 7.1.0?

[pweijian]

I have been using Splunk Enterprise 7.0.3 to do real-time search alert trigger without any issues previously. Recently, I attempt to upgrade Splunk Enterprise to 7.1.0 and found some weird problem with the alert trigger.

This is the setup I have:
1. Using amqp-ta plugin to consume messages from RabbitMQ into IndexA.
2. An alert trigger running on All Time (real-time) search on IndexA to find newly indexed events.
3. The alert is trigger per result.
4. Each alert has 2 actions: Custom Action to write a result to another RabbitMQ Exchange and also log the event to another index.

Problem:
Whenever a new event is being added to IndexA, it will trigger repeatedly trigger the alert action. All the alert action search result is showing the same event that was added. This trigger will continue infinitely until I disable the Alert.

I'm not sure if there are any changes to the architecture of Real-time search alert trigger from 7.0.3 to 7.1.0. Any help would be greatly appreciated!

[xpac]

I've no ready solution, but did you try changing it to scheduled (every minute) search and see if it still happens?

[pweijian]

Hi xpac, thanks for your suggestion. I did tried that and scheduled search is working fine. But my use-case would need to real-time search as I need alerts to be send out immediately when an event is detected.