topic Re: QUEUES: Best Practices in Getting Data In

What are the Best Practices regarding managing queues size?

verbal_666 — Wed, 08 Feb 2023 19:23:28 GMT

Hallo 😊

About this post,
https://community.splunk.com/t5/Building-for-the-Splunk-Platform/Impact-of-increasing-the-queue-size/m-p/630016#M10927

What's the Best Practices about managing queues size?

Let's talk about servers running only Splunkd (Indexers and HFs) and 16GB of total physical memory.

Thanks.

Re: QUEUES: Best Practices

PaulPanther — Wed, 08 Feb 2023 07:58:44 GMT

As you mentioned in the other thread that the queues are blocked frequently on your indexer and HFs you should think about to scale your infrastrcucture instead of increasing the queues because it does not solve the root cause.

Re: QUEUES: Best Practices

verbal_666 — Wed, 08 Feb 2023 08:12:55 GMT

No, sorry, i did not mention that

INDEXERS manage queues OK, they do not get over 10/20% in a full day, by media is 1/5% per INDEXER.
HF was the issue. Leaving queues by default it can't manage the flows, raising to 2GB it does.

My question is: with 16GB server, is there any risk raising to 2GB per queue, or, maybe, also more over? Best Practices what suggest about it?

For now, with 2GB/queue i got no issues in data flow (and there is much much data!!!).
The only little issue i encountered is that, sometimes, during the day, maybe for some networking problems, i found HF to do much os swap, and, in some cases, kernel killed the splunkd process for full swap usage.