Getting Data In

Bandwidth usage for 200 GB of data from heavy forwarder to indexers


I understand from some of the links that using UFs as intermediate forwarding layer add metadata at stream level while using HFs as intermediate layer add metadata at event level.

What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud?

Please advice.

0 Karma

Splunk Employee
Splunk Employee

Hey @pranitprakash, if they answered your question don't forget to "accept" the answer to award karma points/ close the post.

0 Karma


The blog post Universal or Heavy, that is the question? shows you the bandwidth usage difference between universal and heavy forwarders.

If you use SSL the data is compressed with an approx 10 to 1 ratio, so there will be a drastic difference with SSL on vs SSL off.

The general answer is it depends, however you can refer to this answer about minimum bandwidth or this answer about measuring bandwidth used.

The 200GB/day doesn't mean that much in terms of bandwidth usage, if you had a perfect distribution that would be just over 8GB/hour + overheads or approx 2400 kb/s, and if you multiply that by 8 it would be kilobits per second, however if you use SSL and you take into account the overheads this number will obviously change.
Also there is a good chance you will have more data at particular times (which you can control by throttling the forwarder as per this answer)...

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...