We are seeing the aggregation and parsing queues almost constantly flatlining at a 100% on our HFs. On our indexers the queues are at 0% pretty much all the time, with the occasional spike to 20%.
Would increasing the [queue=parsingQueue] and [queue=aggQueue] in system/local/server.conf help lowering the queue fill ratio? The current settings are the default 6MB for Parsing and 1MB for aggregation.
There are only 250 UFs connected to the HF, but that is expected to increase dramatically.
If increasing the queue size works, by how much should I increase it? Is there a rule of thumb that I haven't seen?
The load on the indexer is low and the queue is formed on the HF side is a problem in the processing on the HF side.
Increasing the size of the queue does not solve the problem. Please first check high load processing.
For example, the transfer rate (maxKBps) may be low.Also, if the parsing process is slow, it is also possible to increase the pipeline.