Are there any logs maintained by the Splunk Universal forwarder in case of log processing failures?

[subramanianers]

Are there any logs maintained by the Splunk Universal forwarder in case of log processing failures? I would like to setup an alert and dashboard for the log processing failures that occur while the Universal forwarder tries  logs to my indexer. I need this for compliance purposes.

I checked out the URL - https://docs.splunk.com/Documentation/Splunk/8.2.4/Troubleshooting/WhatSplunklogsaboutitself, and thought splunkd_stderr.log on my servers would do the trick. But when I checked the corresponding logs, I could see only start and stop messages. Can someone provide me the keyword search and the filename for log processing failures in the Splunk Universal Forwarder?

[PickleRick]

Logs forwarded to _internal can provide some insight into input processing errors or stuff like event breaking but be wary that in case something really breaks down and the forwarder crashes or stops forwarding... you're not getting logs to _internal either. After all, they are the logs from /opt/splunk/var/log/splunk/* forwarded to indexer by the UF itself. So if the UF stops forwarding  it only logs to a local file but doesn't send the events from that file to the upstream indexers.

[subramanianers]

Hi @PickleRick . Thanks for replying. I understand that if something is truly broken in the UF, then logs will not be sent to Splunk. I am worried only about the processing errors or invalidations that might have happen in UF, not the case of UF breaking down completely. Is there a particular log file or query that you could point me to so that I can achieve this use case?

[PickleRick]

Usualy the warnings get logged simply into splunkd.log.

In case of scripted/modular inputs they can however have their own separate log files but typical splunk components log to splunkd.log.