Getting Data In

Blockage of Queue

vishaltaneja070
Motivator

Hello All,

Some of the queues are getting blocked in Splunk. Need help to solve it.
alt text

0 Karma

user789
New Member

How did you generate this? I would like to do the same thing.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

While looking at graph, your indexing queue is blocking continuously but percentage is low, for that you are hitting IOPS issue. Have a look at very good white paper created by @dpaper_splunk for disk diagnostics.

For parsing and Aggregation queue, it looks like due to full aggregation queue & back-pressure, parsing queue also filled 100%. I'll suggest you to find which host,source ingested more data during that time and see any splunkd.log warning or error during same time like: timestamp parsing issue. Have a look at detailed pipeline diagram on https://wiki.splunk.com/Community:HowIndexingWorks and if possible configure TIMESTAMP parameter for larger datasets so that splunk parse those data quickly which will help to remediate blocking queue issue.

0 Karma
Get Updates on the Splunk Community!

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...