Hi, I am facing an issue while searching for logs from Search heads. Below are the errors
Distributed: Unable to distribute to peer named INDEXER1 at uri https://10.x.x.1:8089 because replication was unsuccessful. ReplicationStatus: Failed - Failure info: Dispatch Command: Search bundle throttling is occurring because the limit for number of bundles with pending lookups for indexing has been exceeded. This could be the result of large lookup files updating faster than Splunk software can index them. Throttling ends when this instance has caught up with indexing of lookups. If you see this often, contact your Splunk administrator about tuning lookup sizes and max_memtable_bytes.. Please verify connectivity to the search peer, that the search peer is up, and an adequate level of system resources are available. See the Troubleshooting Manual for more information.
Distributed: Unable to distribute to peer named INDEXER2 at uri https://10.x.x.2:8089 because replication was unsuccessful. ReplicationStatus: Failed - Failure info: Dispatch Command: Search bundle throttling is occurring because the limit for number of bundles with pending lookups for indexing has been exceeded. This could be the result of large lookup files updating faster than Splunk software can index them. Throttling ends when this instance has caught up with indexing of lookups. If you see this often, contact your Splunk administrator about tuning lookup sizes and max_memtable_bytes.. Please verify connectivity to the search peer, that the search peer is up, and an adequate level of system resources are available. See the Troubleshooting Manual for more information.
I have increased max_memtable_bytes value in limits.conf from 10MB to 500MB on search heads keeping in mind that this would impact search performance
About our architecture:
• Search head1: Installed Palo Alto, AWS and Linux Apps
• On Search head2: Installed Enterprise Security App
• Two clusterd indexers with SF=1 and RF=2
• 2 non clustered search heads
Thanks!
