We recently installed Splunk Universal forwarder 9.3.2 on Windows 2019 server. After starting the forwarder I see below error in the splunkd.log. Tried uninstalling and installing the UF but still the same error. Please let me know how to fix it.
Error :
02-25-2025 14:52:06.747 -0600 WARN TcpOutputProc [12132 parsing] - The TCP output processor has paused the data flow. Forwarding to host_dest=(ip of indexer) inside output group splunkcloud_ from host_src=(ip folder source) has been blocked for blocked_seconds=5600. This can stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.
Hi @Vin
Its worth checking the expiry on your SSL certificates. I have seen cases like this before where something running stops working during an upgrade, when infact its simply a Splunk restart that broke it - Basically if a certificate expires then Splunk can fail to initiate a new connection and therefore will hang on to an existing, established connection.
Use "openssl x509 -in <pathToYourSSL.crt> -noout -text" to validate that your client certificate on your forwarder is still valid.
If that looks fine then its worth having a deeper dive into the splunkd.log ($SPLUNK_HOME/var/log/splunk/splunkd.log) to check for errors when the blocking starts - is there anything here relating to SSL or port in-accesibility.
Were any other changes made around this time? E.g. Host level firewall etc as part of the upgrade?
Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards
Will