How to observe Splunk Forwarder "backlog"?

[jm_tesla]

Hi, suppose a server with Splunk Forwarder on it, where lots of logs that haven't yet shipped to Splunk. Is there any way to get an output which lists the files/dirs, the current status (e.g. 50% sent to Splunk), etc.? I know I can see a list of files which are being monitored, but I'd like to get an idea of how much data the forwarded has yet to ship.

[PickleRick]

It's a bit more complicated than that.

Forwarder has (oversimplifying a bit) inputs, outputs and some queueing and buffering mechanics in between. Some inputs can (depending on their configuration) block or not if they have nowhere to send to for further processing because, for example, the output isn't connected to anything and internal queues and buffers are full. Some input's can't (there's no possibility to block, for example, udp packets received from external sources).

Typically file inputs block (it doesn't make much sense configuring them otherwise usually) of they have nowhere to send events downstream. But events already read don't have to be immediately sent to downstream receiver(s). They might be held in forwarder buffer.

If you want to check the file inputs configuration and their state, do

splunk list monitor

and

splunk list inputstatus