Getting Data In

Role of the following Queues

lpolo
Motivator

I have not been able to find any document about the role of the queues listed below just partial information.
Is there a document that explains the role of each queue?

Thanks,
Lp

Queue:
aeq
aggqueue
aq
auditqueue
fschangemanager_queue
indexqueue
nullqueue
parsingqueue
splunktcpin
stashparsing
tcpin_queue
typingqueue

Thanks,
Lp

Tags (2)
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

The Splunk on Splunk app contains a diagram on the indexing page under 'learn more about this view' that shows how data moves through queues. That might be useful.

Also, you can take a look at this:

http://splunk-base.splunk.com/answers/7076/questions-about-splunk-queues

It is also discussed here:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Datapipeline

View solution in original post

romantercero
Path Finder

Your dead on. I also have not been able to find an explanation of what all of those queues do. Not even in the architect training course. I've narrowed most down but I'm still missing an explanation for the typing, stashparsing, parsing, aeq and audit queues

These are the ones I've been able to pinpoint so far with respect to their place in the data pipeline:

Inputs: tcpin, splunktcpin, fschangemanager, exec
Parsing: parsing???, stashparsing???
Merging: agg
Typing: Typing??
Indexing: index

So the issue of course is that you see a specific queue spike up and block but since its function is not documented you don't know the reason so you have to investigate elsewhere whats going on. Sometimes you don't find out until the issue get so bad that you see a direct correlation between a spike in a queue and an acute symptom in one of Splunk's many functions.

Why isn't this documented properly?

zur
Engager

Hi,
Im not sure if it still relevant but I found to change the "tcpin" queue size you need to add queueSize in the inputs.conf under the tcp, so its probably the same deal for "udpin".
Zur

0 Karma

raziasaduddin
Path Finder

I still see these other queues in the splunkd logs and do not know the correct camel case for them so I can't set the queue size higher:

I am most interested in udpin.

[queue=AEQ]
[queue=aggQueue]
[queue=auditQueue]
[queue=exec]
[queue=fschangemanager_queue]
[queue=indexQueue]
[queue=nullQueue]
[queue=parsingQueue]
[queue=stashParsing]
[queue=tcpin_queue]
[queue=typingQueue]
[queue=udpIn]
[queue=wel_queue]
[queue=WEVT]
[queue=winParsing]

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

The Splunk on Splunk app contains a diagram on the indexing page under 'learn more about this view' that shows how data moves through queues. That might be useful.

Also, you can take a look at this:

http://splunk-base.splunk.com/answers/7076/questions-about-splunk-queues

It is also discussed here:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Datapipeline

View solution in original post

Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!