Getting Data In

What is the difference between the current_size_kb vs current_size in metrics.log for a universal forwarder?

RecoMark0
Path Finder

Hello,

I don't quite understand the difference between the current_size_kb value and current_size value in the metrics.log for a Universal Forwarder. This is for the parsingqueue, as I am getting the "Could not send data to output queue" error in splunkd.log sometimes.

Metrics.log example line:

05-05-2016 19:03:06.209 +0000 INFO  Metrics - group=queue, name=parsingqueue, max_size_kb=1228800, current_size_kb=1226982, current_size=19790, largest_size=19815, smallest_size=19692

So current_size_kb is almost the same size as my max_size_kb, but then current_size is only ~20MB. I started my maxSize value for parsingQueue in the server.conf at like 200MB, and every time I increased it, current_size_kb would increase as well.

Do I need to increase maxSize even more? Or should I just pay attention to current_size?

Thank you

0 Karma
1 Solution

splunk_force_as
Path Finder

I believe the current_size=19790, largest_size=19815, and smallest_size=19692 are all referencing the number of events in a queue. Where as *_kb is referencing RAM size. My assumption is based on the entry below:

maxSize = [|[KB|MB|GB]]
* Specifies default capacity of a queue.
* If specified as a lone integer (for example, maxSize=1000), maxSize
indicates the maximum number of events allowed in the queue.
* If specified as an integer followed by KB, MB, or GB (for example,
maxSize=100MB), it indicates the maximum RAM allocated for queue.
* The default is 500KB.

View solution in original post

ddrillic
Ultra Champion

The following page speaks a bit about it - About metrics.log

It mentions current_size but not current_size_kb. It says -

-- Most of these values are not interesting. But current_size, especially considered in aggregate, across events, can tell you which portions of Splunk indexing are the bottlenecks. If current_size remains near zero, then probably the indexing system is not being taxed in any way. If the queues remain near 1000, then more data is being fed into the system (at the time) than it can process in total.

0 Karma

splunk_force_as
Path Finder

I believe the current_size=19790, largest_size=19815, and smallest_size=19692 are all referencing the number of events in a queue. Where as *_kb is referencing RAM size. My assumption is based on the entry below:

maxSize = [|[KB|MB|GB]]
* Specifies default capacity of a queue.
* If specified as a lone integer (for example, maxSize=1000), maxSize
indicates the maximum number of events allowed in the queue.
* If specified as an integer followed by KB, MB, or GB (for example,
maxSize=100MB), it indicates the maximum RAM allocated for queue.
* The default is 500KB.

Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...