Getting Data In

Parsing Queue blocked on Heavy Forwarder

omuelle1
Communicator

Hey guys,

I got some question regarding parsing queue issues I have been observing on our Heavy Forwarders. I am currently seeing between 500 and 1000 blocked events on each heavy forwarder daily when running:

index=_internal host=HF blocked=true

The total ratio of blocked events seems to be about 10% and they mostly all seem to appear in the aggqueue:

alt text

My main question is if this is reason for concern or what the impact on my current Splunk environment would be. Also why would all this blocking be in mainly one queue ?

Thank you,

Oliver

0 Karma
1 Solution

codebuilder
SplunkTrust
SplunkTrust

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

----
An upvote would be appreciated and Accept Solution if it helps!

View solution in original post

0 Karma

codebuilder
SplunkTrust
SplunkTrust

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

anmolpatel
Builder

The issue appears to be at the aggQueue based on the screenshot.
So check if the props.conf is configured correctly for the sourcetypes.
Things you want to check:
- Should_Linemerge
- Max_events
- Time_prefix
- Time_format
- Datetime_config
- Max_Days_Ago

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...