https://community.splunk.com/t5/Monitoring-Splunk/PipelineComponent-CallbackRunnerThread-is-unusually-busy/m-p/546954#M4959 <P>Hi Splunker,</P><P>I found the following abnormal problems when checking the Splunk system</P><P>&nbsp;</P><P>&nbsp;</P><LI-CODE lang="markup">04-07-2021 05:54:18.961 +0800 INFO PipelineComponent - CallbackRunnerThread is unusually busy, this may cause service delays: time_ms=0 new=0 null=0 total=2119 {'name':'DistributedRestCallerCallback','valid':'1','null':'0','last':'3','time_ms':'0'},{'name':'HTTPAuthManager:timeoutCallback','valid':'1','null':'0','last':'1','time_ms':'0'},{'name':'HttpInputMetrics:timerCallback','valid':'2','null':'0','last':'2113','time_ms':'0'},{'name':'IndexProcessor:ipCallback-0','valid':'1','null':'0','last':'6','time_ms':'0'},{'name':'MetricsManager:probeandreport','valid':'1','null':'0','last':'0','time_ms':'0'},{'name':'PullBasedPubSubSvr:timerCallback','valid':'1','null':'0','last':'2','time_ms':'0'},{'name':'ThreadedOutputProcessor:timerCallback','valid':'1','null':'0','last':'5','time_ms':'0'},{'name':'triggerCollection','valid':'2111','null':'0','last':'2118','time_ms':'0'} 04-07-2021 05:55:19.961 +0800 INFO PipelineComponent - CallbackRunnerThread is unusually busy, this may cause service delays: time_ms=0 new=0 null=0 total=2119 {'name':'DistributedRestCallerCallback','valid':'1','null':'0','last':'3','time_ms':'0'},{'name':'HTTPAuthManager:timeoutCallback','valid':'1','null':'0','last':'1','time_ms':'0'},{'name':'HttpInputMetrics:timerCallback','valid':'2','null':'0','last':'2113','time_ms':'0'},{'name':'IndexProcessor:ipCallback-0','valid':'1','null':'0','last':'6','time_ms':'0'},{'name':'MetricsManager:probeandreport','valid':'1','null':'0','last':'0','time_ms':'0'},{'name':'PullBasedPubSubSvr:timerCallback','valid':'1','null':'0','last':'2','time_ms':'0'},{'name':'ThreadedOutputProcessor:timerCallback','valid':'1','null':'0','last':'5','time_ms':'0'},{'name':'triggerCollection','valid':'2111','null':'0','last':'2118','time_ms':'0'}</LI-CODE><P>&nbsp;</P><P>&nbsp;</P><P>I don't know what caused it.Which part should I start to find out the reason.</P> Wed, 07 Apr 2021 01:29:33 GMT hellosplunkit 2021-04-07T01:29:33Z https://community.splunk.com/t5/Monitoring-Splunk/PipelineComponent-CallbackRunnerThread-is-unusually-busy/m-p/546954#M4959 <P>Hi Splunker,</P><P>I found the following abnormal problems when checking the Splunk system</P><P>&nbsp;</P><P>&nbsp;</P><LI-CODE lang="markup">04-07-2021 05:54:18.961 +0800 INFO PipelineComponent - CallbackRunnerThread is unusually busy, this may cause service delays: time_ms=0 new=0 null=0 total=2119 {'name':'DistributedRestCallerCallback','valid':'1','null':'0','last':'3','time_ms':'0'},{'name':'HTTPAuthManager:timeoutCallback','valid':'1','null':'0','last':'1','time_ms':'0'},{'name':'HttpInputMetrics:timerCallback','valid':'2','null':'0','last':'2113','time_ms':'0'},{'name':'IndexProcessor:ipCallback-0','valid':'1','null':'0','last':'6','time_ms':'0'},{'name':'MetricsManager:probeandreport','valid':'1','null':'0','last':'0','time_ms':'0'},{'name':'PullBasedPubSubSvr:timerCallback','valid':'1','null':'0','last':'2','time_ms':'0'},{'name':'ThreadedOutputProcessor:timerCallback','valid':'1','null':'0','last':'5','time_ms':'0'},{'name':'triggerCollection','valid':'2111','null':'0','last':'2118','time_ms':'0'} 04-07-2021 05:55:19.961 +0800 INFO PipelineComponent - CallbackRunnerThread is unusually busy, this may cause service delays: time_ms=0 new=0 null=0 total=2119 {'name':'DistributedRestCallerCallback','valid':'1','null':'0','last':'3','time_ms':'0'},{'name':'HTTPAuthManager:timeoutCallback','valid':'1','null':'0','last':'1','time_ms':'0'},{'name':'HttpInputMetrics:timerCallback','valid':'2','null':'0','last':'2113','time_ms':'0'},{'name':'IndexProcessor:ipCallback-0','valid':'1','null':'0','last':'6','time_ms':'0'},{'name':'MetricsManager:probeandreport','valid':'1','null':'0','last':'0','time_ms':'0'},{'name':'PullBasedPubSubSvr:timerCallback','valid':'1','null':'0','last':'2','time_ms':'0'},{'name':'ThreadedOutputProcessor:timerCallback','valid':'1','null':'0','last':'5','time_ms':'0'},{'name':'triggerCollection','valid':'2111','null':'0','last':'2118','time_ms':'0'}</LI-CODE><P>&nbsp;</P><P>&nbsp;</P><P>I don't know what caused it.Which part should I start to find out the reason.</P> Wed, 07 Apr 2021 01:29:33 GMT https://community.splunk.com/t5/Monitoring-Splunk/PipelineComponent-CallbackRunnerThread-is-unusually-busy/m-p/546954#M4959 hellosplunkit 2021-04-07T01:29:33Z https://community.splunk.com/t5/Monitoring-Splunk/PipelineComponent-CallbackRunnerThread-is-unusually-busy/m-p/553039#M5032 <P>Hi&nbsp;<a href="https://community.splunk.com/t5/user/viewprofilepage/user-id/185234">@hellosplunkit</a>&nbsp;Did you ever figure out what caused this error?</P><P>I see it on one busy indexer.</P> Wed, 26 May 2021 00:32:10 GMT https://community.splunk.com/t5/Monitoring-Splunk/PipelineComponent-CallbackRunnerThread-is-unusually-busy/m-p/553039#M5032 burwell 2021-05-26T00:32:10Z