Why would thruput metrics be missing from some forwarders?

[gabriel_vasseur]

I'm using the following search to measure throughput of each forwarder:

index=_internal sourcetype=splunkd metrics source=*metrics.log group=per_sourcetype_thruput | stats sum(kb) as kb by host

This works well generally, but there is one forwarder for which this gives nothing. I really don't understand why.

We do get its metrics logs in _internal, but they have no thruput logs. The docs say:

Thruput is measured in the indexing pipeline. If your data is not reaching this pipeline for some reason, it will not appear in this data.

I don't understand what that means. The forwarder in question is a universal forwarder. We have very similar forwarders for which we have no problem, so I can't really understand what's different.

Can anyone shed some light on this?

[nickhills]

If a forwarder is under pressure (poor network quality, hitting throughput rate limiting, or generally swamped) It will prioritize your monitored data over sending its own _internal logs, which includes metrics etc.

This can also happen if the indexing tier is under similar pressure, but since you only have one host with this issue, that seems less likely.

[gabriel_vasseur]

Interesting, but like you said unlikely to be the case here. We have PS on site this week so if there is time I'll ask them to take a look and will update this question if anything transpires.!