Our environment consists of 1 indexer and 1 search head. Our indexer is currently indexing close to 400GB per day, since we are catching up on historical data. In another week, this should reduce to about 20GB per day.
Meanwhile, we are running a few saved searches on the search head, which would normally run for a few hours. However, we always see the error:
Timed out waiting for peer xx-xxxx-xxx. If this occurs frequently, receiveTimeout in distsearch.conf may need to be increased. Search results might be incomplete!
I have increased receiveTimeout to 900s. I am planning on adding the following stanza to distsearch.conf to reduce the knowledge bundle size:
[replicationWhitelist]
allConf = *.conf
allSpec = *.spec
I know that there may be network issues that are causing the problem. Are there any commands I can use to check the network health between the search head and indexer?
Any other suggestions to avoid this message would be welcome.
It turns out the problem was the nature of the query itself. Since the query was searching for sparse events, there were cases where it would run for 900s, and actually have nothing to return.
Increasing the timeout to a much larger value solved the problem
It turns out the problem was the nature of the query itself. Since the query was searching for sparse events, there were cases where it would run for 900s, and actually have nothing to return.
Increasing the timeout to a much larger value solved the problem
Could you please specify which timeout setting did you increase?
You're digging up an 8 years old thread. I wouldn't expect an answer from its original participants...
Have you covered the basics with ping, traceroute, and telnet?