Amount of data leaving Splunk Heavy Forwarder

[eddieddieddie]

I'm trying to measure the amount of data leaving a heavy forwarder (called HVYFWD01).

This is in preparation to moving to cloud Splunk and the need to get an idea of how much traffic must leave our internal network. Currently all traffic from universal forwarders first goes to the heavy forwarder (then via Cribl) to the indexer.

I tried the following query which uses the tcpout_connections data from the metrics.log file on the heavy forwarder:

index=_internal host=HVYFWD01 sourcetype=splunkd component=Metrics group=tcpout_connections
| stats sum(eval(kb/1024)) AS MB_out by name

This gives results but the figures are about 3 times bigger than the stats Cribl is able to produce for inputs and output. I know there is network overhead which pads out the data a bit - but the numbers I'm getting seem far too big: the above query gives a figure of 600gb, but Cribl records only receiving about 250gb.

[gcusello]

Hi @eddieddieddie ,

sorry, only one question: do you want to migrate to cloud all traffic or only a part of it, maintaining a part of traffic on on-premise indexers?

if all traffic, you don't need to know how much traffic passes through an HF, but you must know the daily license consuption of your infrastructure.

Anyway, you could also assign an index time field to the HF and measure the number of events traffic through your HF, but, in my opinion, the first solution is the most correct.

Ciao.

Giuseppe