splunk 6.6.4 search head >> indexer peer down!!!

[kedjjang]

[ Environment ]
SPLUNK 6.6.4 Search Head Cluster 3 EA
SPLUNK 6.6.4 Indexer Cluster 4EA

Server specifications are very good.

[distsearch.conf] - searcher
connectionTimeout , sendTimeout , receiveTimeout
[limits.conf] - searcher , indexer
base_max_searches ,max_rt_search_multiplier , max_searches_per_cpu

We have set this option to a value that is better than the default value.

If there are many searches on the search server, the indexer is not actually dead, but the error is still happening.

[error msg]
Unable to distribute to peer named * at uri=* using the uri-scheme=https because peer has status="Down". Please verify uri-scheme, connectivity to the search peer, that the search peer is up, and an adequate level of system resources are available. See the Troubleshooting Manual for more informati

[internal log]
ERROR StreamedSearch - sid=remote_adminadmin___RMD561dff7a2ba7b014f_1525229108.3657_03943F0E-D90E-4C81-BA9B-D2F89B841083, Broken pipe

WARN HttpListener - Socket error from ... while accessing /services/streams/search: Broken pipe

Do you know the cause of the symptom and the solution?

[xpac]

I'd guess that the indexer is under such heavy load that the connection times out.

Please show us your actual "better values" for the settings above, and also take a close look at your monitoring console about Ressource usage etc.

[kedjjang]

Thank you for your comment.

distsearch.conf has been doubled from its default value.

limits.conf The default has been increased by a factor of two or three, and the resources have not changed much.

However, the same symptoms.