Getting Data In

Bandwidth usage for 200 GB of data from heavy forwarder to indexers


I understand from some of the links that using UFs as intermediate forwarding layer add metadata at stream level while using HFs as intermediate layer add metadata at event level.

What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud?

Please advice.

0 Karma

Splunk Employee
Splunk Employee

Hey @pranitprakash, if they answered your question don't forget to "accept" the answer to award karma points/ close the post.

0 Karma


The blog post Universal or Heavy, that is the question? shows you the bandwidth usage difference between universal and heavy forwarders.

If you use SSL the data is compressed with an approx 10 to 1 ratio, so there will be a drastic difference with SSL on vs SSL off.

The general answer is it depends, however you can refer to this answer about minimum bandwidth or this answer about measuring bandwidth used.

The 200GB/day doesn't mean that much in terms of bandwidth usage, if you had a perfect distribution that would be just over 8GB/hour + overheads or approx 2400 kb/s, and if you multiply that by 8 it would be kilobits per second, however if you use SSL and you take into account the overheads this number will obviously change.
Also there is a good chance you will have more data at particular times (which you can control by throttling the forwarder as per this answer)...

0 Karma
Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...