Splunk Enterprise

Are there any logs maintained by the Splunk Universal forwarder in case of log processing failures?

subramanianers
Loves-to-Learn Lots

Are there any logs maintained by the Splunk Universal forwarder in case of log processing failures? I would like to setup an alert and dashboard for the log processing failures that occur while the Universal forwarder tries  logs to my indexer. I need this for compliance purposes.

I checked out the URL - https://docs.splunk.com/Documentation/Splunk/8.2.4/Troubleshooting/WhatSplunklogsaboutitself, and thought splunkd_stderr.log on my servers would do the trick. But when I checked the corresponding logs, I could see only start and stop messages. Can someone provide me the keyword search and the filename for log processing failures in the Splunk Universal Forwarder?

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Logs forwarded to _internal can provide some insight into input processing errors or stuff like event breaking but be wary that in case something really breaks down and the forwarder crashes or stops forwarding... you're not getting logs to _internal either. After all, they are the logs from /opt/splunk/var/log/splunk/* forwarded to indexer by the UF itself. So if the UF stops forwarding  it only logs to a local file but doesn't send the events from that file to the upstream indexers.

0 Karma

subramanianers
Loves-to-Learn Lots

Hi @PickleRick . Thanks for replying. I understand that if something is truly broken in the UF, then logs will not be sent to Splunk. I am worried only about the processing errors or invalidations that might have happen in UF, not the case of UF breaking down completely. Is there a particular log file or query that you could point me to so that I can achieve this use case?

Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Usualy the warnings get logged simply into splunkd.log.

In case of scripted/modular inputs they can however have their own separate log files but typical splunk components log to splunkd.log.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...