topic Re: Parsing Queue blocked on Heavy Forwarder in Getting Data In

Parsing Queue blocked on Heavy Forwarder

omuelle1 — Thu, 12 Mar 2020 13:17:03 GMT

Hey guys,

I got some question regarding parsing queue issues I have been observing on our Heavy Forwarders. I am currently seeing between 500 and 1000 blocked events on each heavy forwarder daily when running:

index=_internal host=HF blocked=true

The total ratio of blocked events seems to be about 10% and they mostly all seem to appear in the aggqueue:

My main question is if this is reason for concern or what the impact on my current Splunk environment would be. Also why would all this blocking be in mainly one queue ?

Thank you,

Oliver

Re: Parsing Queue blocked on Heavy Forwarder

anmolpatel — Wed, 30 Sep 2020 04:34:23 GMT

The issue appears to be at the aggQueue based on the screenshot.
So check if the props.conf is configured correctly for the sourcetypes.
Things you want to check:
- Should_Linemerge
- Max_events
- Time_prefix
- Time_format
- Datetime_config
- Max_Days_Ago

Re: Parsing Queue blocked on Heavy Forwarder

codebuilder — Mon, 16 Mar 2020 22:39:28 GMT

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf