I would guess that your configuration is bringing in all AD groups for the organization. What we follow to to limit that is enforce a naming standard for AD groups that need access to splunk, In our instance, have the word Splunk in the AD group name. And then in the configuration, we filter groups having Splunk . This significantly decreases the number of AD groups that the system has to deal with.
By default the limit is 1000. You can view more details of this in the following answer - https://answers.splunk.com/answers/666277/ldap-with-more-then-1000-groups.html
Regarding the error, I have seen that occur when the system is at capacity. I would check resource utilization on your search head cluster. If you have high number of scheduled searches running, this might be eating up your CPU. Typically one search utilizes one CPU core.
... View more