Getting Data In

Monitoring linux forward to ensure communication

timmy13
Communicator

We encountered a forwarder that was not sending data to the indexer nor phoning home. This forwarder's data is a dependancy of many other processes. We found that the splunkd processes were orphaned, killed them, and indexing began working. Is there a way to monitor that splunk is actually sending data?

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Are you really actually looking to see if data is being sent ? Or are you maybe more interested in whether or not its being received ?

I would use the

Distributed Management Console

alt text


Then check out Indexing Performance

alt text


Or check out this answer, where they suggest going to

http://54.183.100.158:8000/en-US/app/search/indexing_volume

in order to split by host.


Or maybe you'd rather go old school, ssh into your indexer, and

sudo tcpdump port 9997

where 9997 is the port that you are forwarding to.


Or maybe you'd just like to use the _internal index. Or you could check splunkd.log.

See this answer for quite a few more options.


A link from the Wiki on troubleshooting forwarding.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...