"not a splunk server" status for (previously) functioning splunk server

[twinspop]

2 Splunk 4.1.3 indexers, 1 4.1.3 search head. The search head is connected to the 2 indexers over a T1 that can get busy. Once every couple of days, I can no longer search on one or both of the indexers via the search head. Status in the distributed search management area says "not a splunk server." If I go into the peer, re-enter the username and password, and save it, everything syncs back up and I'm good to go again. However, that's problematic for unattended uses (monitoring scripts, for example).

Any solutions?

[Mick]

Refreshing the distributed search connection via the manager page will get the servers talking again, but that’s not a permanent fix, it will probably come back again.

The underlying problem is that we don’t deal with network latency very well, but it has been fixed and no complaints yet in 4.1.5. Upgrading is your best bet for a permanent solution

[sdwilkerson]

Thanks Mick. We just upgraded all of the systems here and dist-search re-established cleanly and no problems so far.

[Ledion_Bitincka · Accepted solution]

This is a known issue in 4.1.x and we're working on fixing it in our next maintenance release 4.1.5

[sdwilkerson]

Besides going to 4.1.5, is there any fix (or work-around) for SPL-31279 on 4.1.4?