Stopping Splunk is taking very long to stop

[snosurfur]

Stopping splunkd is taking up to 6 minutes to complete.  We have a process that snapshots the instance and we are stopping splunkd prior to taking that snapshot.  Previously with v9.0.1 we did not experience this; now we are on v9.2.1.

While shutting down I am monitoring spklunkd.log and the only errors I am seeing has to do with the HFs.  'TcpInputProc [65700 tcp] - Waiting for all connections to close before shutting down TcpInputProcessor '.

Has anyone else experienced something similar post upgrade?

[hrawat_splunk]

---

@snosurfur wrote:

Stopping splunkd is taking up to 6 minutes to complete. 

 with the HFs.  'TcpInputProc [65700 tcp] - Waiting for all connections to close before shutting down TcpInputProcessor '.

Has anyone else experienced something similar post upgrade?

---

Anything changed on sending (UF/HF) side?
HF(receiver) waits for sender to disconnect gracefully before it force terminates connections after waiting for ~110 sec(default).

[isoutamo]

Are there lately added more UFs and/or HFs to send logs to your indexer?

[splunkreal]

Hello, is it indexer? If yes it can happen, if it's search head you can force stopping current searches.