Logs compression in Splunk HEC spring boot

priyanka2887 — Fri, 14 Jun 2024 14:04:06 GMT

Hi Team,

Can we compress the logs using Splunk HEC HttpEventCollectorLogbackAppender? Please guide here, how to compress the logs using splunk hec configuration in logback.

Re: Logs compression in Splunk HEC spring boot

tscroggins — Sat, 15 Jun 2024 14:29:10 GMT

Hi @priyanka2887,

At which layer? TLS? HTTP? Splunk?

TLS compression is largely deprecated, vulnerable to well-known attacks, and not (as far as I know) available in core JDK implementations of TLS 1.2+.

HttpEventCollectorLogbackAppender's underlying HTTP implementation, OkHttp, should compress any payload over 1024 bytes by default. See https://github.com/square/okhttp/blob/master/okhttp/src/main/kotlin/okhttp3/OkHttpClient.kt.

HttpEventCollectorLogbackAppender doesn't expose a method or property to modify the threshold. See https://github.com/splunk/splunk-library-javalogging/blob/main/src/main/java/com/splunk/logging/HttpEventCollectorLogbackAppender.java and https://github.com/splunk/splunk-library-javalogging/blob/main/src/main/java/com/splunk/logging/HttpEventCollectorSender.java.

If you want to add support for modifying the compression threshold, see the Contributing section at https://github.com/splunk/splunk-library-javalogging/blob/main/README.md.

Raw data is always compressed in Splunk, although the algorithm is configurable. See the journalCompression setting in https://docs.splunk.com/Documentation/Splunk/latest/Admin/Indexesconf.

topic Re: Logs compression in Splunk HEC spring boot in Getting Data In

Logs compression in Splunk HEC spring boot

Re: Logs compression in Splunk HEC spring boot