Getting Data In

What happens when queues are blocked?

New Member

Hi Guys,

We had a series of events that meant our SUFs were unable to forward to their respective indexers for about 10-days.

For this whole time, the queues were blocked.

Unfortunately, for those 10-days, it appears we have lost data :(.

I have read up a bit about queues, but one thing I am unsure about is what happens when the queues are blocked?

And in my situation, where communication was lost for an extended period of time, do events cache on local disk, if so, for how long? I checked limits.conf but couldn't find much, apart from maxqueuesize.

I guess I need to find the root cause, and future mitigation, of the data loss for the higher-ups. Cheers.

0 Karma

Splunk Employee
Splunk Employee

Unfortunately, the default local disk queue defined is incredibly small. So if the data isn't sent, the data in the queue is FIFO'd. You could increase the disk queue so you have much more local caching until the indexer comes up. But then you should be cognizant of the fact that at that point, once the indexer comes up, will the forwarder be able to catch up. Something to consider.

0 Karma


Hi Dimitri,

I am also facing the similar issue . But dont know how to deal with it. Do you have stepwise procedure to go through to get the solutions OR exactly to ensure the exact reason of the problem.

0 Karma

New Member

I read up on persistent queues and have a good idea of those now.

Unfortunately, we had not configured persistent queues 😞 So looks like our data is gone.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...