Forwarder Output Compression Ratio- what is the expected bandwidth saving of a compressed stream if i activate it?

[Matthias_BY]

Hello,

i can activate compression on the universal forwarder to the indexer. as i understand from the documentation and some answers entries the compression is different between ssl encryption and not.

compressed = [true|false]
* Applies to non-SSL forwarding only. For SSL useClientSSLCompression setting is used.
* If true, forwarder sends compressed data.
* If set to true, the receiver port must also have compression turned on (in its inputs.conf file).
* Defaults to false.

my question: what is the expected bandwidth saving of a compressed stream if i activate it? (useClientSSLCompression and non encrypted?

is it 1:10?

br
matthias

[Matthias_BY]

thanks a lot for all your valuable feedback. Hexx received the points with clear numbers - of course depending on the content within a log it may vary a little bit - but if it is going over wan it's definitely worth to enable compression and even better SSL compression. 😉

[hexx · Accepted solution]

What @bwooden and @Dimitri McKay is all true. Also, here are some rough numbers for compression ratio based on internal testing.

1. Universal forwarder / uncooked data

   • no compression 1:1
   • "native" compression 1:8
   • SSL compression 1:14

2. Regular forwarder / cooked data

   • no compression 1:1
   • "native" compression 1:2
   • SSL compression 1:8

[adobrzeniecki_s]

Hey Hexx, is there a specific doc that shows these compression ratios? This information is fantastic! Would also like a link to the docs page if you have one. Thank you so much.

[andygerber]

This is an old post, but as far as I can see still accurate.

Testing from an HF (azure data) to an indexer cluster, turning on SSL resulted in a 1:8 compression for very short (~65Byte) events.

[bwooden]

Compression is achieved via zlib. Per Dimitri's answer, compression is variable based on content.

Additional information may be found here: http://splunk-base.splunk.com/answers/63384/what-kind-of-compression-is-used-between-forwarders-and-...

[Dimitri_McKay]

That would be contingent upon the repetition and amount of empty space in the data to be compressed.