Getting Data In

Forwarder refusing to start

Path Finder

My forwarder was working fine but stopped and I can't get it running again. Running the splunk start command appears to be working fine but then it fails at the last step.

alt text

No logs are being generated in splunkd.log but some logs are generated in splunkd-utility.log:

alt text

I am not certain what is causing it from starting. Any help would be appreciated. Thanks

0 Karma

SplunkTrust
SplunkTrust

Hi @mawomommoh,

Check the following,

  • Do you have any crash log in splunk log directory?
  • Directory permissions are intact.
  • System has enough memory.
  • Check in windows events to see if there any issues related to splunk crash
0 Karma

Path Finder

Okay. I checked my log files again and I can see a bunch of errors and warnings from before I started encountering the issue:

  • Processing server from outpus.conf: can't resolve a valid IP address for host=XXXX
  • Cooked connection to ip=XXXX timed out
  • Connection to XXXX closed. Read error. An existing connection was forcibly closed by remote host.
  • The TCP processor has paused the data flow. Forwarding to output group default-autolb-group has been blocked for 2300 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.

I have checked my splunk server and deleted/created port 9997 which I am using for the forwarding but still no difference.

0 Karma

SplunkTrust
SplunkTrust

Hi @mawomommoh,

Check your index configuration (host and ip) in output conf of forwarder and make sure that they are reachable and not blocked by firewall or acl

0 Karma