Getting Data In

Parsing Queue blocked on Heavy Forwarder

omuelle1
Communicator

Hey guys,

I got some question regarding parsing queue issues I have been observing on our Heavy Forwarders. I am currently seeing between 500 and 1000 blocked events on each heavy forwarder daily when running:

index=_internal host=HF blocked=true

The total ratio of blocked events seems to be about 10% and they mostly all seem to appear in the aggqueue:

alt text

My main question is if this is reason for concern or what the impact on my current Splunk environment would be. Also why would all this blocking be in mainly one queue ?

Thank you,

Oliver

0 Karma
1 Solution

codebuilder
SplunkTrust
SplunkTrust

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

----
An upvote would be appreciated and Accept Solution if it helps!

View solution in original post

0 Karma

codebuilder
SplunkTrust
SplunkTrust

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

anmolpatel
Builder

The issue appears to be at the aggQueue based on the screenshot.
So check if the props.conf is configured correctly for the sourcetypes.
Things you want to check:
- Should_Linemerge
- Max_events
- Time_prefix
- Time_format
- Datetime_config
- Max_Days_Ago

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>