I can't get fully results in distributed search.

[yutaka1005]

When I have searched in search head, following message was displayed.

error: Some events cannot be displayed because they cannot be fetched from the remote search peer(s). This is likely caused by the natural expiration of the related remote search jobs. To view the omitted events, run the search again.

Also, recently I feel search performance is slow.
Then I investigated cause of this problem, and found following log in each search peer.

• WARN SearchResultWorkUnit - timed out, sending keepalive nConsecutiveKeepalive=27 currentSetStart=1548939053.000000
• ERROR SearchResultWorkUnit - Error in transmit, writing to serialized transmit queue terminated.
• Unable to fully write search results because of Broken pipe wrote 0 out of 2630 bytes

What can be considered as this cause other than "Insufficient value of ulimit on Indexer side" and "Network problem"?

Also, if there is a possibility that there is a network problem, will information for determining it be output to the internal log?

If anyone know about it, please tell me...

[woodcock]

Run the Health Checks on your Monitoring Console, it will probably tell you that you have some combination of these 5 problems on your Indexers; fix ALL OF THEM:

1: THP is on
2: ulimits too low
3: Too few cores
4: Too little RAM
5: Too slow disk I/O

[yutaka1005]

I can't find problem with THP and ulimit.
Also cpu usage and memory usage is not too high.

The only point of concern is that the utilization of the partition where the hot and warm data of each Indexer are stored is close to 95%, so is this related?