Splunk Search

Failing manual Splunk-optimize when 'The index processor has paused data flow. Too many tsidx files' with erno 12

chrzz
Observer

Hello

I've started to get this error message:

The index processor has paused data flow. Too many tsidx files in idx=_audit bucket="/opt/splunk/var/lib/splunk/audit/db/hot_v1_13" , waiting for the splunk-optimize indexing helper to catch up merging them. Ensure reasonable disk space is available, and that I/O write throughput is not compromised.

I then tried the manual splunk-optimize, but that returned this error message:

tm= 1568090447 ERROR merge failed for path=/opt/splunk/var/lib/splunk/audit/db/hot_v1_13 rc=-2 wrc=-2 errno=12 file=/opt/splunk/var/lib/splunk/audit/db/hot_v1_13/1567134306-1567134305-16403447236531428423.tsidx hint=_init_reader_helper in _merge_all_postings_n]
tm= 1568090447 ERROR optimize finished: failed, see rc for more details, dir=/opt/splunk/var/lib/splunk/audit/db/hot_v1_13, rc=-2 (unsigned 254), errno=12
tm= 1568090447 INFO  exiting splunk-optimize process with rc=-2 (unsigned 254)

I've tried to search for "errno=12", but I can't find any info regarding it (just other error numbers).

All my indexes have default settings.

Any suggestions?

0 Karma
1 Solution

lapplander
Explorer

If you run into a situation of too many tsidx files and splunk can't resolve it by it self. Try restarting splunk with splunk restart Upon starting splunk again, all hot buckets will be rolled to warm and the tsidx files gets merged into fewer files.

Watch the stdout from the restart comand to see if it throws any errors and review $SPLUNK_HOME/var/log/splunk/splunkd.log and lookout for any WARN or ERROR messages following the last shutdown.

View solution in original post

0 Karma

GalaxySplunker
Engager

Hello,

I'm encountering the exact same error, only for another index. And it happens a lot, nearly on a daily basis (work days)

Is there any new update on this topic please ? 

Thank you for your help

0 Karma

lapplander
Explorer

If you run into a situation of too many tsidx files and splunk can't resolve it by it self. Try restarting splunk with splunk restart Upon starting splunk again, all hot buckets will be rolled to warm and the tsidx files gets merged into fewer files.

Watch the stdout from the restart comand to see if it throws any errors and review $SPLUNK_HOME/var/log/splunk/splunkd.log and lookout for any WARN or ERROR messages following the last shutdown.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...