All Apps and Add-ons

Splunk Stream: Why does my heavy forwarder not show up in the Distributed Forwarder Management?

wweiland
Contributor

Hi,

I'm about to pull what little hair I have left out. I have a SH and Indexer Cluster running 6.5.1. My cluster uses our own SSL certs for server.conf, web.conf, and inputs.conf, which appear to be working fine. I've installed Splunk Steam (splunk_app_stream and Splunk_TA_stream) on my deployment/admin server. I've installed Splunk_TA_stream on my indexers and a heavy forwarder. I set the location of my server running the splunk_app_stream in the inputs.conf and the Splunk_TA_stream on the heavy forwarder. My problem is that the heavy forwarder still does not show up in the Distributed Forwarder Manager even though I see 2 way traffic via tcpdump. Can anyone help me who has set this up before? What information do you need?

Thank you so much in advance,
Todd

0 Karma
1 Solution

vshcherbakov_sp
Splunk Employee
Splunk Employee

Hi @wweiland,

Sorry to hear about your troubles with Stream..

What OS is your heavy forwarder running on? What's the Stream forwarder config there? Have you run ./set_permissions.sh script (assuming it's *nix)?

Do you have anything suspicious in $SPLUNK_HOME/var/log/splunk/streamfwd.log file on the heavy forwarder? Do you have _internal index on heavy being forwarded from HFW to IDX?

View solution in original post

0 Karma

vshcherbakov_sp
Splunk Employee
Splunk Employee

Hi @wweiland,

Sorry to hear about your troubles with Stream..

What OS is your heavy forwarder running on? What's the Stream forwarder config there? Have you run ./set_permissions.sh script (assuming it's *nix)?

Do you have anything suspicious in $SPLUNK_HOME/var/log/splunk/streamfwd.log file on the heavy forwarder? Do you have _internal index on heavy being forwarded from HFW to IDX?

0 Karma

wweiland
Contributor

It appears that there is stuff in _internal that is absolutely necessary for this app to work properly. I had not set it to forward to the indexers yet.

Thanks

0 Karma

gawilliams
Explorer

Wow thanks for this input! We had to use our Heavy Forwarder to manage stream configurations because we have a Search Head cluster which doesn't support global tokens (as far as we can tell). Once we turned on and configured distributed search on our Heavy Forwarder to the indexer cluster, the Stream Forwarder Management started working! Wish they would document this in the Splunk docs (as far as I can tell it's not called out).

0 Karma

gjanders
SplunkTrust
SplunkTrust

gawilliams on any documentation page you can hit the submit feedback button and they will usually update the documentation !

0 Karma

coltwanger
Contributor

What was in _internal that was necessary? I am forwarding _internal from all of my hosts, but I am experiencing the same issues where my forwarders do not show up under Distributed Forwarder Management. 6.5.1 environment also.

0 Karma

vshcherbakov_sp
Splunk Employee
Splunk Employee

check if you're getting sourcetype="stream:stats" events in the _internal index on the SH - this is what Stream UI requires

coltwanger
Contributor

Thanks for the response! I actually found out I had an issue in some CONF that was preventing me from accessing the endpoint, but all is well now 🙂

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...