Hi Splunk Folks,
We have Splunk Physical Servers with 8GB disk space storage for /opt folder which frequently reaching 90% of the disk space threshold (7.2GB). Since we cannot easily upgrade the disk space because these are Physical servers, we are looking for files that we can remove or migrate.
We found this "/opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/save" folder (1GB in size) that seems like containing the same files (btree_index.dat, btree_records.dat and snapshot) with its predecessor folder (/opt/splunk/var/lib/splunk/fishbucket/splunk_private_db)
Are questions are, what are these Splunk files do and does it safe if we will delete or move them to another folder to free some disk space on /opt?
Here is the commands we used to check which file has consume a large volume of diskspace
-bash-4.2$ df -h /opt/splunk
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-opt 8.0G 6.5G 1.6G 82% /opt
-bash-4.2$ du -h --max-depth=1 /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db
1001M /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/save (Has the most consumed diskspace)
335M /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/snapshot
1.7G /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/ (Total)
If we look inside the "save" folder from /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db, we can see it has same files (btree_index.dat, btree_records.dat and snapshot) . Thus it just might be a backup of splunk_private_db
-bash-4.2$ ls -l /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db
-rw-------. 1 splunk splunk 104865400 Jun 24 04:52 btree_index.dat
-rw-------. 1 splunk splunk 246211800 Jun 24 04:56 btree_records.dat
drwx------. 3 splunk splunk 79 Jun 24 04:49 save
drwx------. 2 splunk splunk 70 Jun 24 04:49 snapshot
-bash-4.2$ ls -l /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/save
-rw-------. 1 splunk splunk 152715440 Nov 22 2019 btree_index.dat
-rw-------. 1 splunk splunk 371572840 Nov 22 2019 btree_records.dat
drwx------. 2 splunk splunk 70 Nov 22 2019 snapshot
Regards,
John Kevin Aracan
