Getting Data In

When sending data with http event collector is there any throttling by default?

andyy5
New Member

Hello,

When sending data with HEC to Splunk Enterprise/Cloud, is there any throttling by default? Or is there an option to set this?
I plan on sending data every few seconds for a long period of time.

Thanks.

0 Karma

ivanreis
Builder

I did not play around the HEC for Splunk cloud. Per my research the HEC is enabled by default for your Splunk Cloud environment with a 1 MB size limit on the maximum content length.

For further information about the cloud limits and constraints, check the link below:
https://docs.splunk.com/Documentation/SplunkCloud/latest/Service/SplunkCloudservice#Service_limits_a...

If you need to increase the capacity
https://docs.splunk.com/Documentation/SplunkCloud/latest/Service/SplunkCloudservice#Subscription_exp...

For Splunk Enterprise there are two parameter that you have to play around. They are "queueSize" and "persistentQueueSize" that you have to setup at inputs.conf under http stanza like this one: [http://yourtokenname]

queueSize The maximum size of the input queue in memory. The value of this parameter is in the form [KB|MB|GB]. The default value is 500KB.

persistentQueueSize = [KB|MB|GB|TB]
* Maximum size of the persistent queue file.
* Persistent queues can help prevent loss of transient data. For information on
persistent queues and how the 'queueSize' and 'persistentQueueSize' settings
interact, search the online documentation for "persistent queues"..
* If you set this to a value other than 0, then 'persistentQueueSize' must
be larger than either the in-memory queue size (as defined by the
'queueSize' setting in inputs.conf or 'maxSize' settings in [queue] stanzas
in server.conf).
* Default: 0 (no persistent queue).

https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/UseHECusingconffiles#Per-token_settings

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...