topic Re: Indexqueue blocked on Heavy Forwarder in Getting Data In

Indexqueue blocked on Heavy Forwarder

cmlombardo — Fri, 05 Jan 2024 17:58:49 GMT

I know there are similar posts about this, but I am not sure on what to do or tweak here.

Messages I am getting are similar to this:
01-05-2024 09:35:07.049 -0800 INFO Metrics - group=queue, ingest_pipe=1, name=indexqueue, blocked=true, max_size_kb=500, current_size_kb=499, current_size=815, largest_size=1764, smallest_size=0

I already set

parallelIngestionPipelines = 2

Also, there is no indication of resource exhaustion on these Heavy Forwarders. CPU is constantly below 25% and RAM is low as well.

What else can I check/do/configure to avoid this?
Also, what happens to the data when this happens?

Thank you!

Re: Indexqueue blocked on Heavy Forwarder

isoutamo — Fri, 05 Jan 2024 18:11:07 GMT

when indexqueue has blocked on HF (or other instances) you should tart to looking from next host which is receiving those events. Quite often the real issue (if there is any issue) are found from it. Just use MC to look how those queues and pipelines are working on it. Usually it’s not an issue, if those queue is locked time by time.
r. Ismo

Re: Indexqueue blocked on Heavy Forwarder

richgalloway — Fri, 05 Jan 2024 18:23:23 GMT

Queues become blocked when the corresponding pipeline is too slow to keep up with incoming data. In this case, the index pipeline is unable to send data out as fast as it's coming in. Verify the HF's destinations are all up, listening, and reachable.

Re: Indexqueue blocked on Heavy Forwarder

cmlombardo — Fri, 05 Jan 2024 18:45:27 GMT

Thank you for your comments. I had the feeling this might be a problem upstream but I wanted to make sure.