Splunk Enterprise Security: Why are notable events not created and notable alert_action is not triggered?

[kwchang_splunk]

One of my Splunk Enterprise Security customer's complained that sometimes the notable events are not created even when the corresponding raw data is there.
So I checked the scheduler log and found that there were cases that the "notable" alert_action was not triggered when the result_count was not "0".

Please see the red box in the screen capture below. This correlation search runs every 2 minutes. At 13:00:54, result_count was "1", but "notable" alert_action was not triggered.

This problem (missing notable events) happens randomly across almost all correlation searches they have, several times in a day.
Unfortunately, I couldn't find any clue from the splunkd log so far. There were no error or warning messages when these problems happened.

My customer is running ES 4.1.1 on Splunk 6.4.3. (search head clustering + multisite indexer clustering environment)

Any suggestion for troubleshooting would be very appreciated.
Thank you in advance.

[smoir_splunk]

Check the throttling of notable events for that search. If the search is throttling notable events, and the "randomly" skipped notable events fall into that time window, that would explain why a notable event isn't being created.

[aglinowi]

Did you find a resolution to this problem? I'm experiencing a very similar issue. Thanks

[kwchang_splunk]

Hi smoir. Thank you for your answer.
Already checked that. Throttling was not configured.

[kwchang_splunk]

Hi Smoir. I just found an ongoing support case which has very similar symptom with my case. I think, I need to contact support team. Thank you.