Getting Data In

Bandwidth usage for 200 GB of data from heavy forwarder to indexers

pranitprakash
Explorer

Hello,
I understand from some of the links that using UFs as intermediate forwarding layer add metadata at stream level while using HFs as intermediate layer add metadata at event level.

What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud?

Please advice.

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee

Hey @pranitprakash, if they answered your question don't forget to "accept" the answer to award karma points/ close the post.

0 Karma

gjanders
SplunkTrust
SplunkTrust

The blog post Universal or Heavy, that is the question? shows you the bandwidth usage difference between universal and heavy forwarders.

If you use SSL the data is compressed with an approx 10 to 1 ratio, so there will be a drastic difference with SSL on vs SSL off.

The general answer is it depends, however you can refer to this answer about minimum bandwidth or this answer about measuring bandwidth used.

The 200GB/day doesn't mean that much in terms of bandwidth usage, if you had a perfect distribution that would be just over 8GB/hour + overheads or approx 2400 kb/s, and if you multiply that by 8 it would be kilobits per second, however if you use SSL and you take into account the overheads this number will obviously change.
Also there is a good chance you will have more data at particular times (which you can control by throttling the forwarder as per this answer)...

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...