topic Re: What are the various queues in Splunk in Deployment Architecture

What are the various queues in Splunk

adityapavan18 — Thu, 11 Apr 2013 11:12:02 GMT

I found the following configuration in my indexers

[queue]
maxSize = 500KB

[queue=AQ]
maxSize = 10MB

[queue=WEVT]
maxSize = 5MB

[queue=aggQueue]
maxSize = 1MB

[queue=fschangemanager_queue]
maxSize = 5MB

[queue=parsingQueue]
maxSize = 6MB

Can anyone help me in understanding each of those queues?

Out of those which one is indexing queue?

Re: What are the various queues in Splunk

kristian_kolb — Thu, 11 Apr 2013 11:31:26 GMT

Read on. I believe that none of them are the indexingQueue.

http://splunk-base.splunk.com/answers/3250/what-does-the-queue-named-aq-how-did-it-get-blocked

http://splunk-base.splunk.com/answers/7076/questions-about-splunk-queues

http://wiki.splunk.com/Community:HowIndexingWorks <-- Good overview, but rather detailed

http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Usepersistentqueues

Re: What are the various queues in Splunk

ChrisG — Thu, 11 Apr 2013 15:43:31 GMT

The overview of the data pipeline and queues in the documentation is in the topic "How data moves through Splunk" in the Distributed Deployment Manual. See also inputs.conf. But I don't see anything in the spec files about these specific settings, so the Answers links that Kristian provided are your best bet. Are you having issues with blocked queues, or are you just learning and discovering?

Re: What are the various queues in Splunk

adityapavan18 — Fri, 12 Apr 2013 09:41:43 GMT

In the splunk environment i am working on, I am losing data(i.e the events coming from Universal forwarder to Indexer), wanted to know if i am losing data because of queues being full.

Re: What are the various queues in Splunk

adityapavan18 — Fri, 12 Apr 2013 09:42:01 GMT

Thanks a lot, links were really helpful