Getting Data In

What happens when queues are blocked?

New Member

Hi Guys,

We had a series of events that meant our SUFs were unable to forward to their respective indexers for about 10-days.

For this whole time, the queues were blocked.

Unfortunately, for those 10-days, it appears we have lost data :(.

I have read up a bit about queues, but one thing I am unsure about is what happens when the queues are blocked?

And in my situation, where communication was lost for an extended period of time, do events cache on local disk, if so, for how long? I checked limits.conf but couldn't find much, apart from maxqueuesize.

I guess I need to find the root cause, and future mitigation, of the data loss for the higher-ups. Cheers.

0 Karma

Splunk Employee
Splunk Employee

Unfortunately, the default local disk queue defined is incredibly small. So if the data isn't sent, the data in the queue is FIFO'd. You could increase the disk queue so you have much more local caching until the indexer comes up. But then you should be cognizant of the fact that at that point, once the indexer comes up, will the forwarder be able to catch up. Something to consider.

0 Karma


Hi Dimitri,

I am also facing the similar issue . But dont know how to deal with it. Do you have stepwise procedure to go through to get the solutions OR exactly to ensure the exact reason of the problem.

0 Karma

New Member

I read up on persistent queues and have a good idea of those now.

Unfortunately, we had not configured persistent queues 😞 So looks like our data is gone.

0 Karma
Get Updates on the Splunk Community!

Observability Unveiled: Navigating OpenTelemetry's Framework and Deployment Options

Observability Unveiled: Navigating OpenTelemetry's Framework and Deployment Options A recent Tech Talk, ...

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...