Knowledge Management

Splunk Input Persistent Queue

hrawat
Splunk Employee
Splunk Employee

https://docs.splunk.com/Documentation/SplunkCloud/9.1.2312/Data/Usepersistentqueues

Persistent queuing is available for certain types of inputs, but not all.

One major limitation with persistent queue at inputs,  enabled on certain UF/HF/IHF/IUF inputs, if downstream parsingqueue/indexqueue/tcpoutqueue are blocked/saturated and a DS bundle push triggers splunk restart, events will be dropped since UF/HF/IHF/IUF failed to drain queues.

On windows DC, persistent queuing is enabled for windows modular inputs, DS bundle push triggers DC restart and still windows modular input events in parsingqueue/tcpoutqueue will be dropped.

On windows DC, some windows event (event occurred while the workstation was being shut down ) logs are always lost.

When Laptops are off the network and restarted/shutdown, in-memory queue events are dropped. 

With PQ at inputs, during splunk restart on forwarding tier, still splunk in-memory queued events might get dropped. 

Typical steps for laptop where events are always lost.
1. Splunk is installed on a Windows Laptop
2. Put the laptop to Sleep
3. The Splunk service will stop, then
4. There will be 1 or 2 Windows events such as 4634-Session_Destroyed.
5. Later the Laptop "wakes up" and there will be 1 or 2 events generated such as 4624-Login
6. Then Splunk service start.
7. The events that were created when sleep started and when sleep ended were not ingested.

Tags (1)
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...