Splunk Forwarder issue

[saotaigiri]

A forwarder which was working before has stopped for up to a month now. After checking, it is confirmed that the forwarder is correctly configured, the service is running and reporting to the indexer, however there is no data showing on the indexer side or when a search is run on splunk  for that forwarder. Can someone please help me in the right direction?

[burwell]

I had those happen today. Restarting the forwarder fixed the problem.

[thambisetty]

Can you check the servername in system/local/server.conf and host in system/local/inputs.conf 

The host and servername from Splunk config should match with server hostname.

and also take the guid from $SPLUNK_HOME/etc/instance.cfg and check in Search head to see how many forwarders are reporting with the GUID.

and also identify hostname associated with guid and check if its matching with server hostname.

[isoutamo]

It’s best practices that those three names are same. But those two inside splunk must not to be a server’s hostname. Time by time (e.g. AWS when hostname changes “all” time) is much better to fix it as splunk instance name instead of physical hos/node name. 

[isoutamo]

If you have MC installed then you could look from forwarder monitoring that it is there. If not installed then just look from index=_intenal host=<your forwarder>.

• Is your forwarder UF or HF?
• And which kind of inputs there are configured?
• Was there e.g. OS update or something else before sending stops?
• What you found from forwarders logs ?

r. Ismo