useClientSSLCompression vs compressed on forwarder

[bmacias84]

I've been work ing on configuring compression using SSL communication between FW and Indexers for the last few hours, but was experiancing some isssues. I've since fixed the issue, but was hoping someone could answer a few questions. Reading through the How to docs, inputs.conf, and output.conf documenation I've notice what appears to discrepancies.

First: in Splunk 5.0.1 in the outputs.conf what is the difference between compressed and useClientSSLCompression? I though that useClientSSLCompression must be used when forwarding encrypted data to indexers; however I've noticed that while using this settings the indexer says it expected compression but forward is not configured. If I used compressed in my outputs.conf under my ssl stanza it works just fine. Is useClientSSLCompression depricated or a bug?

Second: In the documentation Configure_your_forwarders_to_use_your_certificates the compressed setting is used and in output.conf documentation under compressed it states *the following Applies to non-SSL forwarding only. For SSL useClientSSLCompression setting is use. Why is that?

Indexer inputs.conf

[SSL]

rootCA = $SPLUNK_HOME\etc\apps\splunk_indexer_ssl\bin\auth\splunkCAcert.pem

serverCert = $SPLUNK_HOME\etc\apps\splunk_indexer_ssl\bin\auth\splunkCombinedCertNoPass.pem

[splunktcp-ssl:9998]

compressed = true

NON-Functioning Forwarder Outputs.conf:

[tcpout]

defaultGroup = ssl-autolb-group

[tcpout:ssl-autolb-group]

autoLB = true

autoLBFrequency = 30

forceTimebasedAutoLB = true

disabled = false

sslCertPath = $SPLUNK_HOME\etc\apps\splunk_fw_ssl\bin\auth\splunkCombinedCertNoPass.pem

sslRootCAPath = $SPLUNK_HOME\etc\apps\splunk_fw_ssl\bin\auth\splunkCAcert.pem

sslVerifyServerCert = false

useClientSSLCompression = true

useACK = true

maxQueueSize = 6MB

server = 192.168.70.161:9998,192.168.70.165:9998

Functioning Forwarder outputs.conf:

[tcpout]

defaultGroup = ssl-autolb-group

[tcpout:ssl-autolb-group]

autoLB = true

autoLBFrequency = 30

forceTimebasedAutoLB = true

disabled = false

sslCertPath = $SPLUNK_HOME\etc\apps\splunk_fw_ssl\bin\auth\splunkCombinedCertNoPass.pem

sslRootCAPath = $SPLUNK_HOME\etc\apps\splunk_fw_ssl\bin\auth\splunkCAcert.pem

sslVerifyServerCert = false

compressed = true

useACK = true

maxQueueSize = 6MB

server = 192.168.70.161:9998,192.168.70.165:9998

[hexx · Accepted solution]

The compressed attribute only matters if you are forwarding without SSL. It determines whether Splunk will or not perform "native" compression on a per-data chunk (UF, LWF) or per-event (HWF) basis for outgoing data. This must be enabled on both ends for things to work.

If you are forwarding with SSL, unless you explicitly set useClientSSLCompression to false, you will automatically benefit from SSL compression over the data stream. This is significantly more efficient than Splunk-native compression and should be favored in the case of bandwidth restrictions between forwarder and indexer.

[hexx]

If you are using SSL, "compressed = true" will have no effect either in inputs.conf or outputs.conf. We are going to amend the spec files for those two configuration files to make that clear.

[mikelanghorst]

So then hexx should compressed not be set on the indexer if you're using ssl? The inputs.conf doc doesn't really mention it, so it's easy to get into this problem.

[bmacias84]

So that not what I have been experiancing. If I use useClientSSLCompression on the forwarder Indexer closes the connection and the HWF say connection timed out. Though ifI use the compressed settings it works just fine. I'll post my conf shortly.

[jworthington_sp]

When configuring SSL for forwarding to indexers, you are correct, you should use the attribute "compressed".

You can also configure SSL for other types of intra-Splunk communication, which is where the "useClientSSLCompression" attribute might be modified (it defaults to true and generally does not need to be modified). You can see more about the types of intra-Splunk configuration in the following topic:

http://docs.splunk.com/Documentation/Splunk/latest/Security/AboutsecuringSplunktoSplunkcommunication

Hope that helps!

[bmacias84]

The communication that I am talking about is in regards to outputs.conf for Splunk SSL encrypted communication. The other Intra-Splunk communications are controlled by the [sslConfig] stanza server.conf. What outher outputs in the outputs.conf file would use useClientSSLCompression?