Splunkforwarder stopped sending data randomly

[qfulgham]

Hey Guys!

So I have 2 forwarders they had stopped sending current data, I looked over the splunk config with a splunk contractor and found their was nothing wrong with the splunk config. So he told me to check and see if there were possibly any firewalls rules blocking traffic, so I went to the FW team and everything seems good from that aspect because out of nowhere one of the servers starts reporting data again, but the other server is still no reporting data......would anybody have any idea what could be wrong?

• I checked to make sure the splunkwarder was running with the ./splunk status
• I made sure the files were being monitored on the forwarder with the ./splunk list monitor command
• I made sure the timestamp was okay by checking the time on the event versus its _time (but the servers are located in eastern time so time format shouldn't be an issue) (And plus the other server started pulling current data, and they have the same splunk config)

Would there be any other thing I should check that I haven't listed above?

Thanks for the help!

[adonio]

hello there,
first check if you can see data from forwarders in index=_internal
if so, it means the forwarders do send data to indexers and therefore check inputs
another option is to follow that article:
http://docs.splunk.com/Documentation/Splunk/6.6.1/Troubleshooting/Cantfinddata
hope it helps