How to detect a deleted log

[pkeller]

I think some of my forwarders may be experiencing cases where logs are being removed before all events have been forwarded. Is there a string to look for in splunkd.log or possibly recommendations for increased logging levels to detect when splunkd encounters a situation where a file it has been monitoring no longer exists?

[Yasaswy · Accepted solution]

Hi,
This is an interesting question and it would be very useful to have something in Splunk to able to spot this. However from Splunk's standpoint there is no end of file as the logs will be continued to be written to unless the file gets rotated or deleted. So a missing file is nothing abnormal. Additionally reading log file should be much faster than data being written to them. So I don't think there is currently any available setting in Splunk that would record the incident where a log file has been deleted before fully read ..... as Splunk would treat a missing log file as completely read (and it's rotated or deleted by a batch process).

The info on what data inputs are being monitored would be available from the rest calls (URI Reference) but I don't think it will be of much help in this case. What makes you think that these log files are not fully read? Are they big files being moved to a specific input location and then deleted by a batch process?