Getting Data In

The index process has paused data flow. Too many tsidx files

hrawat_splunk
Splunk Employee
Splunk Employee

This issue happens when incoming thruput for hotbuckets is faster than splunk optimize can merge tsidx files and   keep the count < 100(hardcoded). If number of tsidx files per hotbucket are >=100, then indexer will apply indexing pause to allow splunk-optimize catch up.

 

Labels (1)
Tags (1)
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

Post 7.2 onwards following config should fix the issue

 

In indexes.conf set

[default]

maxRunningProcessGroups=12

processTrackerServiceInterval=0

Update (11/16/2022) If the issue is still not resolved, increase maxRunningProcessGroups setting.
For future splunk 9.1 release and splunk cloud releases, the workaround is not needed as the issue is fixed now.

View solution in original post

hrawat_splunk
Splunk Employee
Splunk Employee

Post 7.2 onwards following config should fix the issue

 

In indexes.conf set

[default]

maxRunningProcessGroups=12

processTrackerServiceInterval=0

Update (11/16/2022) If the issue is still not resolved, increase maxRunningProcessGroups setting.
For future splunk 9.1 release and splunk cloud releases, the workaround is not needed as the issue is fixed now.

Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...