Knowledge Management

Splunk Input Persistent Queue

hrawat
Splunk Employee
Splunk Employee

https://docs.splunk.com/Documentation/SplunkCloud/9.1.2312/Data/Usepersistentqueues

Persistent queuing is available for certain types of inputs, but not all.

One major limitation with persistent queue at inputs,  enabled on certain UF/HF/IHF/IUF inputs, if downstream parsingqueue/indexqueue/tcpoutqueue are blocked/saturated and a DS bundle push triggers splunk restart, events will be dropped since UF/HF/IHF/IUF failed to drain queues.

On windows DC, persistent queuing is enabled for windows modular inputs, DS bundle push triggers DC restart and still windows modular input events in parsingqueue/tcpoutqueue will be dropped.

On windows DC, some windows event (event occurred while the workstation was being shut down ) logs are always lost.

When Laptops are off the network and restarted/shutdown, in-memory queue events are dropped. 

With PQ at inputs, during splunk restart on forwarding tier, still splunk in-memory queued events might get dropped. 

Typical steps for laptop where events are always lost.
1. Splunk is installed on a Windows Laptop
2. Put the laptop to Sleep
3. The Splunk service will stop, then
4. There will be 1 or 2 Windows events such as 4634-Session_Destroyed.
5. Later the Laptop "wakes up" and there will be 1 or 2 events generated such as 4624-Login
6. Then Splunk service start.
7. The events that were created when sleep started and when sleep ended were not ingested.

Tags (1)
Get Updates on the Splunk Community!

[Upcoming Webinar] Demo Day: Transforming IT Operations with Splunk

Join us for a live Demo Day at the Cisco Store on January 21st 10:00am - 11:00am PST In the fast-paced world ...

New Year. New Skills. New Course Releases from Splunk Education

A new year often inspires reflection—and reinvention. Whether your goals include strengthening your security ...

Splunk and TLS: It doesn't have to be too hard

Overview Creating a TLS cert for Splunk usage is pretty much standard openssl.  To make life better, use an ...