Getting Data In

Forwarder refusing to start

mawomommoh
Path Finder

My forwarder was working fine but stopped and I can't get it running again. Running the splunk start command appears to be working fine but then it fails at the last step.

alt text

No logs are being generated in splunkd.log but some logs are generated in splunkd-utility.log:

alt text

I am not certain what is causing it from starting. Any help would be appreciated. Thanks

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Hi @mawomommoh,

Check the following,

  • Do you have any crash log in splunk log directory?
  • Directory permissions are intact.
  • System has enough memory.
  • Check in windows events to see if there any issues related to splunk crash
Happy Splunking!
0 Karma

mawomommoh
Path Finder

Okay. I checked my log files again and I can see a bunch of errors and warnings from before I started encountering the issue:

  • Processing server from outpus.conf: can't resolve a valid IP address for host=XXXX
  • Cooked connection to ip=XXXX timed out
  • Connection to XXXX closed. Read error. An existing connection was forcibly closed by remote host.
  • The TCP processor has paused the data flow. Forwarding to output group default-autolb-group has been blocked for 2300 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.

I have checked my splunk server and deleted/created port 9997 which I am using for the forwarding but still no difference.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Hi @mawomommoh,

Check your index configuration (host and ip) in output conf of forwarder and make sure that they are reachable and not blocked by firewall or acl

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...