Monitoring linux forward to ensure communication

[timmy13]

We encountered a forwarder that was not sending data to the indexer nor phoning home. This forwarder's data is a dependancy of many other processes. We found that the splunkd processes were orphaned, killed them, and indexing began working. Is there a way to monitor that splunk is actually sending data?

[aljohnson_splun]

Are you really actually looking to see if data is being sent ? Or are you maybe more interested in whether or not its being received ?

I would use the

Distributed Management Console

---

Then check out Indexing Performance

---

Or check out this answer, where they suggest going to

http://54.183.100.158:8000/en-US/app/search/indexing_volume

in order to split by host.

---

Or maybe you'd rather go old school, ssh into your indexer, and

sudo tcpdump port 9997

where 9997 is the port that you are forwarding to.

---

Or maybe you'd just like to use the _internal index. Or you could check splunkd.log.

See this answer for quite a few more options.

---

A link from the Wiki on troubleshooting forwarding.