How to fix a full Splunk indexer queue?

[HX]

Hi, 

one of indexers stops receiving events as the indexer queue is full. I check the splunkd.log, see lots of error message, 

02-09-2023 09:49:02.354 +1100 ERROR pipeline [1807 indexerPipe_1] - Runtime exception in pipeline=indexerPipe processor=indexer error='Unable to create directory /mnt/splunk_index_hot/_internaldb/db/hot_v1_311115391 because Input/output error' confkey='source::/opt/splunk/var/log/splunk/splunkd.log|host::hostname|splunkd|1456359'
02-09-2023 09:49:02.354 +1100 ERROR pipeline [1807 indexerPipe_1] - Uncaught exception in pipeline execution (indexer) - getting next event

Anyone came cross this situation? How to fix it up? Thanks!

[scelikok]

Hi @HX,

It seems either your disk (/mnt/splunk_index_hot) is full or not accessible.