topic Re: "AlertNotifier busy... Consider improving action execution speed or increase action_execution_threads in limits.conf" How to increase the limit? in Alerting

"AlertNotifier busy... Consider improving action execution speed or increase action_execution_threads in limits.conf" How to increase the limit?

lukasz92 — Thu, 27 Oct 2016 08:57:21 GMT

I have very large number (over hundred) of scheduled searches done every minute. Some have alert actions to send an email.

I get thousands of events like this:

WARN SavedSplunker - AlertNotifier busy! Failed to enqueue job for search_id="scheduler_(...)". No actions will be executed. Consider improving action execution speed or increase action_execution_threads in limits.conf

some thousands per day.

I raised this limit to 6 (and to 10 after) - and now I get about 0-100 per day.

How to cope with that? Documentation says, that 10 is the maximum. I want to disable this limit at all.

Re: "AlertNotifier busy... Consider improving action execution speed or increase action_execution_threads in limits.conf" How to increase the limit?

Masa — Tue, 29 Sep 2020 11:41:13 GMT

There is no way to make it unlimited. Good practice is to use more search heads with SHC to distribute alerts and increase actions_queue_size (500 or so? ) Any unlimited settings need to be careful. It could use up all available resources.

Re: "AlertNotifier busy... Consider improving action execution speed or increase action_execution_threads in limits.conf" How to increase the limit?

lukasz92 — Tue, 27 Dec 2016 12:25:53 GMT

There are 4 searchheads in cluster now.

Thank you for this setting - I will try it.