[ Environment ]
SPLUNK 6.6.4 Search Head Cluster 3 EA
SPLUNK 6.6.4 Indexer Cluster 4EA
Server specifications are very good.
[distsearch.conf] - searcher
connectionTimeout , sendTimeout , receiveTimeout
[limits.conf] - searcher , indexer
base_max_searches ,max_rt_search_multiplier , max_searches_per_cpu
We have set this option to a value that is better than the default value.
If there are many searches on the search server, the indexer is not actually dead, but the error is still happening.
Unable to distribute to peer named * at uri=* using the uri-scheme=https because peer has status="Down". Please verify uri-scheme, connectivity to the search peer, that the search peer is up, and an adequate level of system resources are available. See the Troubleshooting Manual for more informati
ERROR StreamedSearch - sid=remote_adminadmin___RMD561dff7a2ba7b014f_1525229108.3657_03943F0E-D90E-4C81-BA9B-D2F89B841083, Broken pipe
WARN HttpListener - Socket error from ... while accessing /services/streams/search: Broken pipe
Do you know the cause of the symptom and the solution?
I'd guess that the indexer is under such heavy load that the connection times out.
Please show us your actual "better values" for the settings above, and also take a close look at your monitoring console about Ressource usage etc.
Thank you for your comment.
distsearch.conf has been doubled from its default value.
limits.conf The default has been increased by a factor of two or three, and the resources have not changed much.
However, the same symptoms.