Getting Data In

Forwarder Phoning Home but No Logs Found

Path Finder

I have all fifty-two forwarders phoning home to the server. However, I can't see any logs from the hosts besides the server itself. So, the computer where the Enterprise is installed has no issues with its logs being captured, but every other host has the issue. I don't believe it's a Firewall complication since I've turned it off.

What I am looking for is some areas to trouble shoot this problem -- any pointers on where to look. I have three apps (_server_app_windowshost, splunk_TA_windows, and uf_outputs) which say they have all been deployed to 100% of the hosts.

I did copy over my uf_outputs and _server_app_windowshost files from my other ntowrk, edditted the IP address where needed, then placed them on the new server under deployment apps.

Any advice on what to look for would be helpful. Thanks.

0 Karma


I'd start by making sure the forwarders are connecting to the indexers, not just the deployment server.

index=_internal source=*metrics.log* tcpin_connections | stats count by sourceIp

You should see connections from the forwarders and if they are connected you should also be able to search _internal for each host since they should be forwarding internal data. If they aren't connected I would check your outputs deployed to the forwarders.

If they are connecting, just not sending the Windows data you are expecting I would check the monitor on the forwarder. On a forwarder you can run "./splunk list monitor" to see which files it is trying to monitor and forward. If what you are looking for isn't listed I'd check the deployment server for the app that defines the inputs and make sure the app is set to trigger a restart when updated. It could be possible the app was deployed but Splunk was not restarted and the forwarder has not yet loaded in the inputs.

0 Karma

Ultra Champion

You better start with I can't find my data!

Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...