SHC Captain Disconnected

[MFiller90]

Hey Splunkers,

It seems that several times per hour that our SHC (of 9 SH's) seems to randomley disconnect the SHC Captain. We only ever see the "pumpkins" in the top on the search head GUIs. Something to the effect of "Search Head Captain disconnected blah blah"

The fun part is that, nothing actually bad happens. Our searches continue to run and complete. The errors eventually disappear after about 45 seconds. The annoying part is that if the timing is right, and you try to push a new bundle via Deployer --> SHC, the Deployer says "No captain found amongst members". To which, we just repush and it magically goes through just fine.

Running Splunk Enterprise v7.0.5

Has anybody ever seen anything similar to this?

Thanks!

[nareshinsvu]

Might be network glitches? Do you see any errors in your splunkd.log?

For detail of troubleshooting, you might want to file a Support case with a splunk diag file so that Support engineer can take look into more detail.