I'm working with Splunk setup to copy and index disk logs from remote servers using scheduled rsync transfer.
The rsync transfer job has a bandwidth limit specified to avoid overloading the remote servers and the Splunk server.
During a recent incident, this rsync bandwidth limit was reached because logs grew too quickly (about 5 GB/hour for about a day). During this time, logs were not transferred for indexing.
This is as designed, but Splunk reports nothing for that timeframe, nor gives an indication that data is missing.
Our Splunk admin says the rsync transfer failures cannot be reported. Is there a way to use Splunk to detect when logs were not indexed as expected?
... View more