https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433416#M20429 <P>Hello,</P> <P>I have recently inherited a Splunk Enterprise (v6.6) instance with some serious issues. The architecture is a distributed one with the Search head, Indexer and Heavy Forwarder all residing on different hosts. The primary problem I am facing is that after a short period of the time the queues (parsing, aggregator, typing and index) reach 100% and result in the error mentioned in the title.</P> <P>Upon investigating the Index file directory where the errors are reported, there are 100+ .lock files that seem to replicate as file.lock, file.lock.lock, file.lock.lock.lock etc etc.</P> <P>The machines that are running Splunk have more than enough RAM,CPU and IOPS. I have manually run splunk-optimize with no effect. I am lost on what to do next and almost considering deleting the index (not preferred) to resolve this issue.</P> <P>Any help would be much appreciated.</P> Wed, 29 Aug 2018 00:03:34 GMT alex387 2018-08-29T00:03:34Z https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433416#M20429 <P>Hello,</P> <P>I have recently inherited a Splunk Enterprise (v6.6) instance with some serious issues. The architecture is a distributed one with the Search head, Indexer and Heavy Forwarder all residing on different hosts. The primary problem I am facing is that after a short period of the time the queues (parsing, aggregator, typing and index) reach 100% and result in the error mentioned in the title.</P> <P>Upon investigating the Index file directory where the errors are reported, there are 100+ .lock files that seem to replicate as file.lock, file.lock.lock, file.lock.lock.lock etc etc.</P> <P>The machines that are running Splunk have more than enough RAM,CPU and IOPS. I have manually run splunk-optimize with no effect. I am lost on what to do next and almost considering deleting the index (not preferred) to resolve this issue.</P> <P>Any help would be much appreciated.</P> Wed, 29 Aug 2018 00:03:34 GMT https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433416#M20429 alex387 2018-08-29T00:03:34Z https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433417#M20430 <P>where are the queues pile up? indexer or heavy forwarder?</P> Wed, 29 Aug 2018 00:34:24 GMT https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433417#M20430 adonio 2018-08-29T00:34:24Z https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433418#M20431 <P>Starts at the Heavy Forwarder and once it is maxed out, it then flows onto the indexer until it is 100% m</P> Wed, 29 Aug 2018 00:40:15 GMT https://community.splunk.com/t5/Deployment-Architecture/Index-Processor-The-index-processor-has-paused-data-flow-Too/m-p/433418#M20431 alex387 2018-08-29T00:40:15Z