Indexqueue blocked on Heavy Forwarder

[cmlombardo]

I know there are similar posts about this, but I am not sure on what to do or tweak here.

Messages I am getting are similar to this:
01-05-2024 09:35:07.049 -0800 INFO Metrics - group=queue, ingest_pipe=1, name=indexqueue, blocked=true, max_size_kb=500, current_size_kb=499, current_size=815, largest_size=1764, smallest_size=0

I already set

parallelIngestionPipelines = 2

Also, there is no indication of resource exhaustion on these Heavy Forwarders. CPU is constantly below 25% and RAM is low as well.

What else can I check/do/configure to avoid this?
Also, what happens to the data when this happens?

Thank you!

[cmlombardo]

Thank you for your comments. I had the feeling this might be a problem upstream but I wanted to make sure.

[richgalloway]

Queues become blocked when the corresponding pipeline is too slow to keep up with incoming data.  In this case, the index pipeline is unable to send data out as fast as it's coming in.  Verify the HF's destinations are all up, listening, and reachable.

[isoutamo]

Hi

when indexqueue has blocked on HF (or other instances) you should tart to looking from next host which is receiving those events. Quite often the real issue (if there is any issue) are found from it. Just use MC to look how those queues and pipelines are working on it. Usually it’s not an issue, if those queue is locked time by time. 
r. Ismo