Alerting

"AlertNotifier busy... Consider improving action execution speed or increase action_execution_threads in limits.conf" How to increase the limit?

Communicator

I have very large number (over hundred) of scheduled searches done every minute. Some have alert actions to send an email.

I get thousands of events like this:

WARN SavedSplunker - AlertNotifier busy! Failed to enqueue job for search_id="scheduler_(...)". No actions will be executed. Consider improving action execution speed or increase action_execution_threads in limits.conf

some thousands per day.

I raised this limit to 6 (and to 10 after) - and now I get about 0-100 per day.

How to cope with that? Documentation says, that 10 is the maximum. I want to disable this limit at all.

0 Karma

Splunk Employee
Splunk Employee

There is no way to make it unlimited. Good practice is to use more search heads with SHC to distribute alerts and increase actionsqueuesize (500 or so? ) Any unlimited settings need to be careful. It could use up all available resources.

0 Karma

Communicator

There are 4 searchheads in cluster now.

Thank you for this setting - I will try it.

0 Karma