How should I configure the Stream app to have streamfwd not use SSL when connecting to HEC endpoints?

[cnestrud]

I am trying to use Splunk Stream with the HTTP Event Collector. I have set HEC to not use SSL. In inputs.conf on the servers with HEC enabled:

[http]
disabled=0
enableSSL=0

The HEC URLs are entered as http://host:port in the Stream app configuration. When the standalone streamfwd is run it gets the correct URLs from the stream app but it looks like it tries to use SSL.

From streamfwd.log:

2017-02-13 15:38:58 WARN 140663285286656 stream.SplunkSenderHTTPEventCollector - (#39) SSL handshake failed: unknown protocol

The HEC endpoints are on different servers from the SHC which has the Stream app. The streamfwd binary is on a separate server without Splunk.

[vshcherbakov_sp · Accepted solution]

You can check the HEC endpoints streamfwd is using by navigating to http://streamfwd-host:8889. By default this endpoint is only accessible from the same host (ie http://localhost:8889), but can be changed by adding ipAddr=0.0.0.0 param to `streamfwd.conf.

This will open an internal debugging/monitoring web interface. From there, click on the [Stats] link on the white splunk box and check the returned JSON file. HEC endpoints are presented as "indexers" array, e.g.:

...
indexers": [
"https://hec-host1:8088",
"https://hec-host2:8088",
...
"https://hec-hostn:8088",
]
}

Check if this data matches the expected protocol/host list.

PS: be careful not to leave it open after you're done with the troubleshooting (for security concerns, etc).