Splunk Search

How to search how much bandwidth a forwarder is using?

sbattista09
Contributor

I'm trying to find how much bandwidth a forwarder is using and how many hosts are sending over the forwarder. I want to show it in a timechart that has the hosts' total bandwidth and then another line that had all the totals. I'm not sure where to start since most documents show using the _internal index. any input will help, thanks!

1 Solution

jbsplunk
Splunk Employee
Splunk Employee

By default, the universal forwarder sends some data to the indexers, and one of these logs is the metrics.log, which you can use to calculate the outgoing thruput rate every 30s.

The way this is done in Splunk on Splunk(you can simply look at the metrics view if you've got that app installed) is like something like this:

index=_internal source=*metrics.log* group=tcpout_connections | eval KBps=tcp_Bps/1024 | timechart sum(KBps) as KBps span=15m by host| eval MB=KBps*_span/1024 | fields - KBps

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

By default, the universal forwarder sends some data to the indexers, and one of these logs is the metrics.log, which you can use to calculate the outgoing thruput rate every 30s.

The way this is done in Splunk on Splunk(you can simply look at the metrics view if you've got that app installed) is like something like this:

index=_internal source=*metrics.log* group=tcpout_connections | eval KBps=tcp_Bps/1024 | timechart sum(KBps) as KBps span=15m by host| eval MB=KBps*_span/1024 | fields - KBps

louismai
Path Finder

index=_internal source=metrics.log group=udpin_connections |timechart avg(udp_kbps) max(udp_kbps) span=5m

UDP is similar. But UDP has _udp_bps=25434.085277542734, _udp_kbps=24.837973903850326, _udp_avg_thruput=15.692977114824794; so we might not need to do calculation.

0 Karma

twinspop
Influencer

You risk double accounting the metrics being logged ON the indexer and the metrics being logged TO the indexer (ON the forwarders). I would encourage you to include a host qualifier in your search.

0 Karma

jkellerman_splu
Splunk Employee
Splunk Employee

Wouldn't avg(KBps) be the correct function here?
Otherwise you would be presenting the sum of the individual measurements during the 15 min span. That would not be per second.

sbattista09
Contributor

jkellerman,
can you show me what you mean, is that not what is in my search?

0 Karma

lqiao
Explorer

Hi sbattista09,

I think jkellerman was referring the answer from jbsplunk that the function in the timechart should be avg instead of sum which I agree.

0 Karma

sowings
Splunk Employee
Splunk Employee

Splunk on Splunk? Don't you mean DMC? 🙂

sbattista09
Contributor

awesome!

I also found this nifty little search-

| rest splunk_server=* /services/server/introspection/indexer | eval status = case(reason == ".", status,   reason == "", status,   isnull(reason), status,   1 == 1, status.": ".reason) | fields splunk_server, average_KBps, status   | eval average_KBps = round(average_KBps, 0) | rename splunk_server as Instance, average_KBps as "Indexing Rate (KB/s)", status as Status, reason as Reason
0 Karma
Get Updates on the Splunk Community!

App Building 101 - Build Your First App!

WATCH RECORDING NOW   Tech Talk: App Dev Edition Splunk has tons of out-of-the-box functionality, and you’ve ...

Introducing support for Amazon Data Firehose in Splunk Edge Processor

We’re excited to announce a powerful update to Splunk Data Management with added support for Amazon Data ...

The Observability Round-Up: September 2024

What’s up Splunk Community! Welcome to the latest edition of the Observability Round-Up, a monthly series in ...