OK. Mind you that those are not directly Splunk-related things, it's more like my personal outlook based on 25 years of admin experience.
1. For me, you're doing too many things. I understand your approach, but I prefer the KISS approach when doing shell scripts. For more complex things I'd go python. But that's my personal taste. I don't like overly complicated bash scripts because they tend to get messy quickly.
To be honest, I would do it completely another way around - do a single simple script to manage single frozen index storage with given parameters (size/time) and possibly add another "managing" script spawning the single script for each index independently.
2. I don't see the point in generating random PROCESS_ID. And even less so - using an external dependency of openssl to generate the value of this variable.
3. You are hardcoding many paths - LOG_FILE, CONFIG_FILE, FROZEN_PATH... It might be ok if you're only writing a one-off script for internal uses. When doing a portable solution it's much more user-friendly to make it configurable. The easiest way would be to externalize those definitions to another file and include from that file using the dot (source) command. Bonus - you can use the same config file in both scripts. Presently you have to configure both scripts separately.
4. Chmod another script so you can run it... that's not nice. It should be in installation instructions.
5. I don't like the idea of a script to create the service file. Just provide a service file template with the instructions to customize it if needed. (I would probably do it with cron instead of a service but that's me - I'm old).
6. IMHO such script manipulating relatively sensitive data should use a lock file to prevent it from being run multiple times in parallel.
7. The mechanics of deleting frozen buckets is highly suboptimal. You're spawning several finds and du after removing each file. That's a lot of unnecessary disk scanning. Also - why removing files from the bucket directory and after that removing an empty directory?
8. To make the script consistent with how Splunk handles buckets you should not use ctime or mtime but rather take the timestamps from the bucket boundaries. (they might result in the same order since probably buckets will be frozen in the same order they should roll out from frozen but - especially if you're using shared storage for frozen across multiple cluster nodes and do deduplication - it's not granted).
9. Sorry to say that but it shows that it was written with ChatGPT - there are some design choices which are inconsistent (like timestamp manipulation and sometimes doing arithmetics using built-in bash functionality whereas other times spawning bc).
So again - I do appreciate the effort. It's just that I would either do it completely differently (which might be simply my personal taste) or - if it was to be a quick and dirty hack - I would simply use tmpreaper (if your distro provides it) or do
find /frozen_path -ctime +Xd -delete
(yes, it doesn't account for size limits, but is quick and reliable)
If you want to use size limits, just list directory sizes, sort by date, sum them up until you hit the limit, delete the rest. Et voila. Honestly, don't overthink it.
