Recently we encountered a problem. /opt file system on the indexer server has reached 100% due to which users were unable to do search.
we found that /opt/splunk/archive/main folder is consuming most of the disk space (499 GB out of 500 GB). This is the folder which contains frozen data. Please find the below line in the configuration of main index in indexes.conf
you can delete frozen data without any problem obiviously you no longer have that data.
If you're not interested to these data, why don't you delete them after retention time instead copying them in frozen dir?
Yes you can always delete frozen data in splunk as in splunk frozen data is not searchable.
But in your case it seems it's not frozen data which occupying space (499 GB out of 500 GB) because 500GB space is considered for hot+warm and cold storage and not frozen storage.
So I think your space under $SPLUNK_DB/defaultdb/ (colddb or db) is getting 499GB full.