Getting Data In

Forwarder Output Compression Ratio

Matthias_BY
Communicator

Hello,

i can activate compression on the universal forwarder to the indexer. as i understand from the documentation and some answers entries the compression is different between ssl encryption and not.

compressed = [true|false]
* Applies to non-SSL forwarding only. For SSL useClientSSLCompression setting is used.
* If true, forwarder sends compressed data.
* If set to true, the receiver port must also have compression turned on (in its inputs.conf file).
* Defaults to false.

my question: what is the expected bandwith saving of a compressed stream if i activate it? (useClientSSLCompression and non encrypted?

is it 1:10?

br
matthias

Tags (2)
1 Solution

hexx
Splunk Employee
Splunk Employee

What @bwooden and @Dimitri McKay is all true. Also, here are some rough numbers for compression ratio based on internal testing.

  1. Universal forwarder / uncooked data

    • no compression 1:1
    • "native" compression 1:8
    • SSL compression 1:14
  2. Regular forwarder / cooked data

    • no compression 1:1
    • "native" compression 1:2
    • SSL compression 1:8

View solution in original post

Matthias_BY
Communicator

thanks a lot for all your valuable feedback. Hexx received the points with clear numbers - of course depending on the content within a log it may vary a little bit - but if it is going over wan it's definitely worth to enable compression and even better SSL compression. 😉

0 Karma

hexx
Splunk Employee
Splunk Employee

What @bwooden and @Dimitri McKay is all true. Also, here are some rough numbers for compression ratio based on internal testing.

  1. Universal forwarder / uncooked data

    • no compression 1:1
    • "native" compression 1:8
    • SSL compression 1:14
  2. Regular forwarder / cooked data

    • no compression 1:1
    • "native" compression 1:2
    • SSL compression 1:8

View solution in original post

bwooden
Splunk Employee
Splunk Employee

Compression is achieved via zlib. Per Dimitri's answer, compression is variable based on content.

Additional information may be found here: http://splunk-base.splunk.com/answers/63384/what-kind-of-compression-is-used-between-forwarders-and-...

Dimitri_McKay
Splunk Employee
Splunk Employee

That would be contingent upon the repetition and amount of empty space in the data to be compressed.

.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!