https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308848#M58115 <P>Hello,<BR /> I understand from some of the links that using UFs as intermediate forwarding layer add metadata at <STRONG>stream level</STRONG> while using HFs as intermediate layer add metadata at <STRONG>event level</STRONG>. </P> <P>What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud? </P> <P>Please advice. </P> Mon, 16 Oct 2017 23:43:32 GMT pranitprakash 2017-10-16T23:43:32Z https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308848#M58115 <P>Hello,<BR /> I understand from some of the links that using UFs as intermediate forwarding layer add metadata at <STRONG>stream level</STRONG> while using HFs as intermediate layer add metadata at <STRONG>event level</STRONG>. </P> <P>What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud? </P> <P>Please advice. </P> Mon, 16 Oct 2017 23:43:32 GMT https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308848#M58115 pranitprakash 2017-10-16T23:43:32Z https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308849#M58116 <P>The blog post <A href="https://www.splunk.com/blog/2016/12/12/universal-or-heavy-that-is-the-question.html">Universal or Heavy, that is the question?</A> shows you the bandwidth usage difference between universal and heavy forwarders.</P> <P>If you use SSL the data is compressed with an approx 10 to 1 ratio, so there will be a drastic difference with SSL on vs SSL off.</P> <P>The general answer is it depends, however you can refer to <A href="https://answers.splunk.com/answers/2014/what-is-the-minimum-network-bandwidth-required-for-splunk-forwarding.html">this answer about minimum bandwidth</A> or <A href="https://answers.splunk.com/answers/340084/how-to-search-how-much-bandwidth-a-forwarder-is-us.html">this answer about measuring bandwidth used.</A></P> <P>The 200GB/day doesn't mean that much in terms of bandwidth usage, if you had a perfect distribution that would be just over 8GB/hour + overheads or approx 2400 kb/s, and if you multiply that by 8 it would be kilobits per second, however if you use SSL and you take into account the overheads this number will obviously change.<BR /> Also there is a good chance you will have more data at particular times (which you can control by throttling the forwarder as per this <A href="https://answers.splunk.com/answers/2014/what-is-the-minimum-network-bandwidth-required-for-splunk-forwarding.html">answer</A>)...</P> Tue, 17 Oct 2017 11:42:56 GMT https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308849#M58116 gjanders 2017-10-17T11:42:56Z https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308850#M58117 <P>Hey @pranitprakash, if they answered your question don't forget to "accept" the answer to award karma points/ close the post.</P> Tue, 17 Oct 2017 14:53:52 GMT https://community.splunk.com/t5/Getting-Data-In/Bandwidth-usage-for-200-GB-of-data-from-heavy-forwarder-to/m-p/308850#M58117 lfedak_splunk 2017-10-17T14:53:52Z