topic What is the difference between the current_size_kb vs current_size in metrics.log for a universal forwarder? in Getting Data In

What is the difference between the current_size_kb vs current_size in metrics.log for a universal forwarder?

RecoMark0 — Tue, 29 Sep 2020 09:36:35 GMT

Hello,

I don't quite understand the difference between the current_size_kb value and current_size value in the metrics.log for a Universal Forwarder. This is for the parsingqueue, as I am getting the "Could not send data to output queue" error in splunkd.log sometimes.

Metrics.log example line:

05-05-2016 19:03:06.209 +0000 INFO  Metrics - group=queue, name=parsingqueue, max_size_kb=1228800, current_size_kb=1226982, current_size=19790, largest_size=19815, smallest_size=19692

So current_size_kb is almost the same size as my max_size_kb, but then current_size is only ~20MB. I started my maxSize value for parsingQueue in the server.conf at like 200MB, and every time I increased it, current_size_kb would increase as well.

Do I need to increase maxSize even more? Or should I just pay attention to current_size?

Thank you

Re: What is the difference between the current_size_kb vs current_size in metrics.log for a universal forwarder?

splunk_force_as — Tue, 29 Sep 2020 09:51:00 GMT

I believe the current_size=19790, largest_size=19815, and smallest_size=19692 are all referencing the number of events in a queue. Where as *_kb is referencing RAM size. My assumption is based on the entry below:

maxSize = [|[KB|MB|GB]]
* Specifies default capacity of a queue.
* If specified as a lone integer (for example, maxSize=1000), maxSize
indicates the maximum number of events allowed in the queue.
* If specified as an integer followed by KB, MB, or GB (for example,
maxSize=100MB), it indicates the maximum RAM allocated for queue.
* The default is 500KB.

Re: What is the difference between the current_size_kb vs current_size in metrics.log for a universal forwarder?

ddrillic — Fri, 03 Jun 2016 02:07:51 GMT

The following page speaks a bit about it - About metrics.log

It mentions current_size but not current_size_kb. It says -

-- Most of these values are not interesting. But current_size, especially considered in aggregate, across events, can tell you which portions of Splunk indexing are the bottlenecks. If current_size remains near zero, then probably the indexing system is not being taxed in any way. If the queues remain near 1000, then more data is being fed into the system (at the time) than it can process in total.