We have a shortage of disk space in one indexer. Can we delete data present in the colddb directory?

[Hemnaath]

We are currently running out of space in one Splunk indexer out of 5 indexers in our distributed environment. Using Splunk 6.2.1 Version.
Total size of the indexer volume is about 5.2TB. Currently we are left out with less then 100 GB of space and everyday an average of 10GB of space is occupied. The data that is occupying space is almost 3.5 year old data. and most of the data is present under the colddb storage unit under the mount point /splogs.

Disk Usage status

df -h /splogs
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_splunk03_san-splunk_logs
                      5.6T  5.3T   93G  99% /splogs

We could find most of the space is occupied by these indexes.

[net_proxy], [net_fw], [unix_svrs] & [unix_bsm] 

Example:

[root@splunk03 splogs]# cd unix_svrs
[root@splunk03 unix_svrs]# ls -ltr
total 416
drwx------    2 splunk splunk   4096 Apr 19  2012 thaweddb
drwx------ 1590 splunk splunk 102400 Aug  6 09:18 colddb
drwx------ 1890 splunk splunk 131072 Aug  6 12:51 summary
drwx------ 1893 splunk splunk 143360 Aug  6 12:53 datamodel_summary
drwx------  307 splunk splunk  28672 Aug  6 12:54 db
[root@splunk03 unix_svrs]# du -sh *
1007G   colddb
1.6G    datamodel_summary
229G    db
366M    summary
4.0K    thaweddb

[root@splunk03 splogs]# cd net_fw
[root@splunk03 net_fw]# ls -ltr
total 612
drwx------    2 splunk splunk   4096 Apr 19  2012 thaweddb
drwx------ 1358 splunk splunk 131072 Sep 27  2015 summary
drwx------ 2956 splunk splunk 180224 Aug  6 12:17 colddb
drwx------ 3258 splunk splunk 266240 Aug  6 12:55 datamodel_summary
drwx------  313 splunk splunk  28672 Aug  6 12:55 db
[root@splunk03 net_fw]# du -sh *
**1.3T**    colddb
76G     datamodel_summary
147G    db
24M     summary
4.0K    thaweddb

Indexes.conf details for these indexers

[volume:Hot]
path = /splogs

[volume:Cold]
path = /splogs

[volume:Base]
path = /splogs

[default]
frozenTimePeriodInSecs = 31536000

[net_fw]
homePath = volume:Hot/net_fw/db
coldPath = volume:Cold/net_fw/colddb
tstatsHomePath = volume:Hot/net_fw/datamodel_summary
thawedPath = $SPLUNK_DB/net_fw/thaweddb
maxTotalDataSizeMB = 250000

[unix_svrs]
homePath = volume:Hot/unix_svrs/db
coldPath = volume:Cold/unix_svrs/colddb
tstatsHomePath = volume:Hot/unix_svrs/datamodel_summary
thawedPath = $SPLUNK_DB/unix_svrs/thaweddb
maxTotalDataSizeMB = 250000

[summary]
frozenTimePeriodInSecs = 188697600

There are other indexers configured in the same manner as shown above in Indexes.conf.

Kindly let me know whether we can delete the data that are present under the colddb directory for the indexer occupying more than 1TB. By doing this, what will be the impact? Or is there any other method we can prevent the failure of the splunk service due to low disk space?