Getting Data In

Why would thruput metrics be missing from some forwarders?

gabriel_vasseur
Contributor

I'm using the following search to measure throughput of each forwarder:

index=_internal sourcetype=splunkd metrics source=*metrics.log group=per_sourcetype_thruput | stats sum(kb) as kb by host

This works well generally, but there is one forwarder for which this gives nothing. I really don't understand why.

We do get its metrics logs in _internal, but they have no thruput logs. The docs say:

Thruput is measured in the indexing pipeline. If your data is not reaching this pipeline for some reason, it will not appear in this data.

I don't understand what that means. The forwarder in question is a universal forwarder. We have very similar forwarders for which we have no problem, so I can't really understand what's different.

Can anyone shed some light on this?

0 Karma

nickhills
Ultra Champion

If a forwarder is under pressure (poor network quality, hitting throughput rate limiting, or generally swamped) It will prioritize your monitored data over sending its own _internal logs, which includes metrics etc.

This can also happen if the indexing tier is under similar pressure, but since you only have one host with this issue, that seems less likely.

If my comment helps, please give it a thumbs up!
0 Karma

gabriel_vasseur
Contributor

Interesting, but like you said unlikely to be the case here. We have PS on site this week so if there is time I'll ask them to take a look and will update this question if anything transpires.!

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...