All Apps and Add-ons

Does the Splunk Universal Forwarder ever throttle its collection of data due to high system / resource utilization?

rjthibod
Champion

Is there any built-in mechanism (e.g. settings in limits.conf or server.conf) that would throttle the execution of the Splunk Universal Forwarder in such a way that it stops collecting perfmon data (via the Splunk Add-on for Microsoft Windows) if the host was under distress with high CPU or high memory utilization?

I am not talking about throttling data output or throughput. This is not a matter of limiting outgoing data.

Instead, I am talking about the collection scripts not running when the system is heavily loaded. I cannot reproduce it on my Windows 7 system, but I seem to recall seeing it a long time ago and someone else is reporting that they have observed this behavior.

The perfmon collections resumes once the system is no longer under distress, so that is why I suspected that there might be some configuration option that tells the forwarder to stop collecting if a certain CPU threshold is reached.

0 Karma
1 Solution

lguinn2
Legend

AFAIK, there is no mechanism to do this, HOWEVER - if the system is overloaded, Splunk may not be able to schedule the search jobs. Splunk considers this "skipping" the jobs, and it is not a good thing. I suspect that this may be the behavior you have seen.
The only thing I can think of is: you can configure the scheduler. But I am not sure this is a good way to accomplish what you want.

View solution in original post

0 Karma

lguinn2
Legend

AFAIK, there is no mechanism to do this, HOWEVER - if the system is overloaded, Splunk may not be able to schedule the search jobs. Splunk considers this "skipping" the jobs, and it is not a good thing. I suspect that this may be the behavior you have seen.
The only thing I can think of is: you can configure the scheduler. But I am not sure this is a good way to accomplish what you want.

0 Karma

rjthibod
Champion

Thanks @lguinn. That is what my suspicion but wanted to see if the community could confirm.

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...