Getting Data In

HttpInputDataHandler parsing error: Server is busy

dilpreetsingh
Engager

Getting this errror in Heavy forwarder logs:

02-06-2020 18:28:00.283 +0000 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997, pset=0, reuse=0.
02-06-2020 18:28:08.696 +0000 ERROR HttpInputDataHandler - Parsing error : Server is busy
02-06-2020 18:28:08.703 +0000 ERROR HttpInputDataHandler - Parsing error : Server is busy
02-06-2020 18:28:09.881 +0000 WARN  TailReader - Could not send data to output queue (parsingQueue), retrying...
02-06-2020 18:28:16.717 +0000 ERROR HttpInputDataHandler - Parsing error : Server is busy
02-06-2020 18:28:16.725 +0000 ERROR HttpInputDataHandler - Parsing error : Server is busy
02-06-2020 18:28:20.075 +0000 WARN  TcpOutputProc - The TCP output processor has paused the data flow. Forwarding to output group US-EAST-1-INDEXER-CLUSTER has been blocked for 10 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.
02-06-2020 18:28:21.717 +0000 ERROR HttpInputDataHandler - Parsing error : Server is busy
02-06-2020 18:28:21.726 +0000 ERROR HttpInputDataHandler - Parsing error : Server is busy
02-06-2020 18:28:24.881 +0000 INFO  TailReader -   ...continuing.
02-06-2020 18:28:29.847 +0000 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997, pset=0, reuse=0.

No error spotted in indexer logs.
Data is being received via HTTP Event Collector on Heavy Forwarder.

0 Karma
1 Solution

nickhills
Ultra Champion

There are a few clues in that log.

Ther first one is this:

TailReader - Could not send data to output queue (parsingQueue)

That could indicate that your HF is recieving data faster than it can send it on to your indexers.

The next warning is therefore also important

TcpOutputProc - The TCP output processor has paused the data flow. Forwarding to output group US-EAST-1-INDEXER-CLUSTER has been blocked for 10 seconds.

That means your indexers (or the network to them) is throttling your dataflow, this could be an issue on the indexers, however if you are seeing no problems on that side, then congestion or load is the likely culpret.

Now we come back to your main question:

ERROR HttpInputDataHandler - Parsing error : Server is busy

In the context of the other two messages (and the absence of problems on the indexers) suggests that you HF is struggeling to keep up, possibly network related, or possibly the rate of messages you are recieving over HEC is greater than you can parse and send for indexing.

You need to look at your queues and performance counters on the HF to establish what course of action to take next.

If my comment helps, please give it a thumbs up!

View solution in original post

nickhills
Ultra Champion

There are a few clues in that log.

Ther first one is this:

TailReader - Could not send data to output queue (parsingQueue)

That could indicate that your HF is recieving data faster than it can send it on to your indexers.

The next warning is therefore also important

TcpOutputProc - The TCP output processor has paused the data flow. Forwarding to output group US-EAST-1-INDEXER-CLUSTER has been blocked for 10 seconds.

That means your indexers (or the network to them) is throttling your dataflow, this could be an issue on the indexers, however if you are seeing no problems on that side, then congestion or load is the likely culpret.

Now we come back to your main question:

ERROR HttpInputDataHandler - Parsing error : Server is busy

In the context of the other two messages (and the absence of problems on the indexers) suggests that you HF is struggeling to keep up, possibly network related, or possibly the rate of messages you are recieving over HEC is greater than you can parse and send for indexing.

You need to look at your queues and performance counters on the HF to establish what course of action to take next.

If my comment helps, please give it a thumbs up!
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...