topic running log on splunk heavy forwarder in Getting Data In

running log on splunk heavy forwarder

trojan_81 — Mon, 16 Mar 2020 19:22:35 GMT

I have a heavy forwarder onprem installed on a windows OS.

I am troubleshooting why logs are not coming into the splunk cloud indexer from a cloud service over API. The api is between my onprem splunk heavy forwarder and the cloud service. I suspect the problem is on the cloud service side. I need a way to tell if the logs are even making it to my heavy forwarder. Is there a way to tail a running log on the heavy forwarder?

Also I am referring to the onprem slunk server as a heavy forwarder. Is that the proper term? It sends data to the cloud indexer.

Re: running log on splunk heavy forwarder

richgalloway — Mon, 16 Mar 2020 20:10:47 GMT

Look in \Program Files\Splunk\var\log\splunk\splunkd.log for errors connecting to Splunk Cloud. They'll probably be associated with the TcpOutputProc component. If the HF has its web server enabled (it is by default) then you can sign in and search for index=_internal (component=TcpOutputProc OR SSL).

Problems connecting to Splunk Cloud are usually on the on-prem side. Firewalls often block connections. Certificates may be missing or in the wrong location. The OS may not support the right version of SSL. The logs should offer suggestions about the cause in your case.

Re: running log on splunk heavy forwarder

trojan_81 — Mon, 16 Mar 2020 20:26:58 GMT

Rich. Other apps are able to get logs into this Forwarder via REST API and the logs are searchable on splunk cloud indexer. That tells me that the Forwarder is probably ok. I just need a way to show proof so that I can go back to the vendor's app side.
Proof will be confirming that the logs are not making it to the Forwarder.

Re: running log on splunk heavy forwarder

richgalloway — Tue, 17 Mar 2020 00:53:42 GMT

I still recommend checking firewalls at both ends. Verify the app has the right URI.

If the HF has its web server enabled (it is by default) then you can sign in and search for index=_internal component=TcpInputProc for errors or warnings about incoming connections.