Splunk Search

Failing manual Splunk-optimize when 'The index processor has paused data flow. Too many tsidx files' with erno 12

chrzz
Observer

Hello

I've started to get this error message:

The index processor has paused data flow. Too many tsidx files in idx=_audit bucket="/opt/splunk/var/lib/splunk/audit/db/hot_v1_13" , waiting for the splunk-optimize indexing helper to catch up merging them. Ensure reasonable disk space is available, and that I/O write throughput is not compromised.

I then tried the manual splunk-optimize, but that returned this error message:

tm= 1568090447 ERROR merge failed for path=/opt/splunk/var/lib/splunk/audit/db/hot_v1_13 rc=-2 wrc=-2 errno=12 file=/opt/splunk/var/lib/splunk/audit/db/hot_v1_13/1567134306-1567134305-16403447236531428423.tsidx hint=_init_reader_helper in _merge_all_postings_n]
tm= 1568090447 ERROR optimize finished: failed, see rc for more details, dir=/opt/splunk/var/lib/splunk/audit/db/hot_v1_13, rc=-2 (unsigned 254), errno=12
tm= 1568090447 INFO  exiting splunk-optimize process with rc=-2 (unsigned 254)

I've tried to search for "errno=12", but I can't find any info regarding it (just other error numbers).

All my indexes have default settings.

Any suggestions?

0 Karma
1 Solution

lapplander
Explorer

If you run into a situation of too many tsidx files and splunk can't resolve it by it self. Try restarting splunk with splunk restart Upon starting splunk again, all hot buckets will be rolled to warm and the tsidx files gets merged into fewer files.

Watch the stdout from the restart comand to see if it throws any errors and review $SPLUNK_HOME/var/log/splunk/splunkd.log and lookout for any WARN or ERROR messages following the last shutdown.

View solution in original post

0 Karma

GalaxySplunker
Engager

Hello,

I'm encountering the exact same error, only for another index. And it happens a lot, nearly on a daily basis (work days)

Is there any new update on this topic please ? 

Thank you for your help

0 Karma

lapplander
Explorer

If you run into a situation of too many tsidx files and splunk can't resolve it by it self. Try restarting splunk with splunk restart Upon starting splunk again, all hot buckets will be rolled to warm and the tsidx files gets merged into fewer files.

Watch the stdout from the restart comand to see if it throws any errors and review $SPLUNK_HOME/var/log/splunk/splunkd.log and lookout for any WARN or ERROR messages following the last shutdown.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...