ZSTD Journal compression working for hot/warm buckets only

[romainbouajila]

Environment : Distributed Splunk Enterprise (indexer cluster)

Version: 9.0.5

Issue: After setting journalCompression to zstd in indexes.conf, we noticed that the setting is applied for warm but not for frozen buckets. The setting was applied months ago. In the following example, we can see that files timestamped from today are zst in warm and gzip in frozen.

I did not find any related information in documentation indexesconf

Is it an expected behavior or am I missing some setting in my configuration?

Evidence:

## WARM BUCKETS

[splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/warm/<index_name>
[...]
drwx--x---. 3 splunk splunk 4096 Apr 30 11:19 db_1714450734_1714041906_2521_1B4FA1BE-AA81-459F-B38A-1FB23A018EDB

[splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/warm/<index_name>/db_1714450734_1714041906_2521_1B4FA1BE-AA81-459F-B38A-1FB23A018EDB/rawdata/
[...]
-rw-------. 1 splunk splunk 113295494 Apr 30 11:19 journal.zst

## FROZEN BUCKETS

[splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/frozen/<index_name>
[...]
drwx------. 3 splunk splunk 29 Apr 30 11:20 rb_1709121660_1709115460_2204_3BF8DDF1-9874-4848-9DB4-880DA5EBA00F

[splunk@indexer (PROD) ~]$ ls -latr /var/lib/splunk/frozen/<index_name>/rb_1709121660_1709115460_2204_3BF8DDF1-9874-4848-9DB4-880DA5EBA00F/rawdata/
[...]
-rw-------. 1 splunk splunk 2342045 Feb 28 19:08 journal.gz

[scelikok]

Hi @romainbouajila,

JournalCompression setting is related to only new created warm buckets. Freezing process just copies warm buckets rawdata from warm folder to frozen folder when their freezing rules valid (size or age).  

In your case it seems your zstd setting applied after 28 Feb. That is why previous created buckets  are gzipped.

You should see zstd files in your frozen buckets after some time.