Developing for Splunk Enterprise

Impact of increasing the queue size?

chintan_shah
Path Finder

I am facing issues while searching any logs and its takes a lot of time to index the log.
While investigating, i am seeing many times the queues are blocked as mentioned below.

INFO Metrics - group=queue, name=indexqueue, blocked=true, max_size_kb=500, current_size_kb=499, current_size=1466, largest_size=1466, smallest_size=0

Is it worth to increase the max size of the queues? Also while config file i need to change to increase the same?

Tags (2)
0 Karma

gjanders
SplunkTrust
SplunkTrust

Increasing the queue size may work to add an additional buffer for bursts of incoming log data, however it is unlikely to fix your blocked indexing queue.

For example if you made you incoming TCP input queue bigger, then more data can queue there while the data is getting written to disk, you can do this with various other queues to allow a bigger buffer as data travels down the queue.

However the only real way to resolve an index queue issue on an indexer would be to index less (i.e. add more indexers) or to have faster I/O (get faster disk for your hot section). if your data is getting forwarded then you would look at the system getting forwarded to as well.

Good luck!

0 Karma

ddrillic
Ultra Champion

We had an interesting discussion about it recently at What's the maxSize we can set for the event-processing queues?

It's crucial to adjust the queue's sizes as you compensate for slow I/O with memory cashing - marvelous!!

Can you please post your indexer's $SPLUNK_HOME/etc/system/local/server.conf?

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @chintan_shah,

Indexqueue blocking is due to many reason.

1.) Storage latency - if indexer is not able to write in storage at required IOPS then indexqueue will be filled. -> You need to check with storage team whether you are getting required IOPS from storage or not.
2.) Universal forwarder sending more logs compare to indexers capacity. -> In this case if you are running Indexer cluster you need to add more indexer in your cluster.

Before increasing any queue size I'll recommnd to contact splunk support.

Thanks,
Harshil

0 Karma

splunk24
Path Finder

i am seeing the above issue for exec queue .. please guide what to do for that and exec queue is for running some scripts or what ?

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>