topic Re: How to search how much bandwidth a forwarder is using? in Splunk Search

How to search how much bandwidth a forwarder is using?

sbattista09 — Fri, 08 Jan 2016 15:45:02 GMT

I'm trying to find how much bandwidth a forwarder is using and how many hosts are sending over the forwarder. I want to show it in a timechart that has the hosts' total bandwidth and then another line that had all the totals. I'm not sure where to start since most documents show using the _internal index. any input will help, thanks!

Re: How to search how much bandwidth a forwarder is using?

jbsplunk — Fri, 08 Jan 2016 16:38:24 GMT

By default, the universal forwarder sends some data to the indexers, and one of these logs is the metrics.log, which you can use to calculate the outgoing thruput rate every 30s.

The way this is done in Splunk on Splunk(you can simply look at the metrics view if you've got that app installed) is like something like this:

index=_internal source=*metrics.log* group=tcpout_connections | eval KBps=tcp_Bps/1024 | timechart sum(KBps) as KBps span=15m by host| eval MB=KBps*_span/1024 | fields - KBps

Re: How to search how much bandwidth a forwarder is using?

sbattista09 — Fri, 08 Jan 2016 19:01:25 GMT

awesome!

I also found this nifty little search-

| rest splunk_server=* /services/server/introspection/indexer | eval status = case(reason == ".", status,   reason == "", status,   isnull(reason), status,   1 == 1, status.": ".reason) | fields splunk_server, average_KBps, status   | eval average_KBps = round(average_KBps, 0) | rename splunk_server as Instance, average_KBps as "Indexing Rate (KB/s)", status as Status, reason as Reason

Re: How to search how much bandwidth a forwarder is using?

sowings — Fri, 08 Jan 2016 19:09:43 GMT

Splunk on Splunk? Don't you mean DMC? 🙂

Re: How to search how much bandwidth a forwarder is using?

jkellerman_splu — Wed, 18 May 2016 13:53:16 GMT

Wouldn't avg(KBps) be the correct function here?
Otherwise you would be presenting the sum of the individual measurements during the 15 min span. That would not be per second.

Re: How to search how much bandwidth a forwarder is using?

sbattista09 — Fri, 15 Jul 2016 14:30:41 GMT

jkellerman,
can you show me what you mean, is that not what is in my search?

Re: How to search how much bandwidth a forwarder is using?

twinspop — Mon, 04 Dec 2017 19:21:38 GMT

You risk double accounting the metrics being logged ON the indexer and the metrics being logged TO the indexer (ON the forwarders). I would encourage you to include a host qualifier in your search.

Re: How to search how much bandwidth a forwarder is using?

lqiao — Thu, 07 Jun 2018 19:37:45 GMT

Hi sbattista09,

I think jkellerman was referring the answer from jbsplunk that the function in the timechart should be avg instead of sum which I agree.

Re: How to search how much bandwidth a forwarder is using?

louismai — Wed, 30 Sep 2020 05:30:54 GMT

index=_internal source=metrics.log group=udpin_connections |timechart avg(udp_kbps) max(udp_kbps) span=5m

UDP is similar. But UDP has _udp_bps=25434.085277542734, _udp_kbps=24.837973903850326, _udp_avg_thruput=15.692977114824794; so we might not need to do calculation.