topic Troubleshooting search in Splunk Search

Troubleshooting search

So76 — Tue, 26 Apr 2022 23:20:48 GMT

I ran this search on splunk cloud web and I got the results below. Can anyone help on how to resolve

index=_internal source=*/splunkforwarder/var/log/splunk/splunkd.log OR source=*SplunkUniversalForwarder\\var\\log\\splunk\\splunkd.log log_level=ERROR | transaction host component

1) 04-26-2022 13:27:26.944 -0700 ERROR ExecProcessor [4000 ExecProcessor] - message from ""c:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" WinEventCommonChannel - EvtDC::connectToDC: DsBind failed: (1722) 04-26-2022 13:27:26.944 -0700 ERROR ExecProcessor [4000 ExecProcessor] - message from ""c:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::init: Failed to bind to DC, dc_bind_time=1031 msec 04-26-2022 13:27:27.959 -0700 ERROR ExecProcessor [4000 ExecProcessor] - message from ""c:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" WinEventCommonChannel - EvtDC::connectToDC: DsBind failed: (1722) 04-26-2022 13:27:29.090 -0700 ERROR ExecProcessor [4000 ExecProcessor] - message from ""c:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" WinEventCommonChannel - EvtDC::connectToDC: DsBind failed: (1722) 04-26-2022 13:27:29.715 -0700 ERROR ExecProcessor [4000 ExecProcessor] - message from ""c:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" WinEventCommonChannel - EvtDC::connectToDC: DsBind failed: (1722)

2) 04-26-2022 09:38:13.402 -0700 ERROR TcpOutputFd [5228 TcpOutEloop] - Connection to host=1*******0.146:9997 failed 04-26-2022 09:38:43.312 -0700 ERROR TcpOutputFd [5228 TcpOutEloop] - Connection to host=1*******0.146:9997 failed 04-26-2022 09:39:13.173 -0700 ERROR TcpOutputFd [5228 TcpOutEloop] - Connection to host=1*******0.146:9997 failed 04-26-2022 09:39:43.118 -0700 ERROR TcpOutputFd [5228 TcpOutEloop] - Connection to host=1*******0.146:9997 failed 04-26-2022 09:40:12.952 -0700 ERROR TcpOutputFd [5228 TcpOutEloop] - Connection to host=1*******0.146:9997 failed

3) 04-26-2022 08:27:54.691 -0700 ERROR PipelineComponent [6004 CallbackRunnerThread] - Monotonic time source didn't increase; is it stuck?

Re: Troubleshooting search

bowesmana — Tue, 26 Apr 2022 23:47:49 GMT

What needs to be resolved?

Re: Troubleshooting search

So76 — Wed, 27 Apr 2022 12:53:41 GMT

How do I resolve these issues below?

c:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" WinEventCommonChannel - EvtDC::connectToDC: DsBind failed

ERROR PipelineComponent [6004 CallbackRunnerThread] - Monotonic time source didn't increase; is it stuck?

Connection to host=1*******0.146:9997 failed 04-26-2022 09:39:13.173 -0700 ERROR TcpOutputFd [5228 TcpOutEloop]

Thanks

Re: Troubleshooting search

PickleRick — Wed, 27 Apr 2022 13:21:25 GMT

Short answer is - find the cause of the failures.

Long answer is - with the first error - the Windows Event Log modular input tries to connect to your domain controller and cannot.

Why it wants to connect? Possibly because you're using

evt_resolve_ad_obj=true

Why it fails? Because the user running the splunk forwarder has no permissions to connect. Most typically - this happens when the forwarder process is run with local account (usually Local System) instead of a domain account.

Second one - well, that's google for you. https://community.splunk.com/t5/Monitoring-Splunk/Has-anyone-seen-this-Error-message-Monotonic-time-source-didn-t/m-p/316324

Third one - the forwarder tried to connect to indexer and failed. Why? You'd have to verify the connectivity and possibly check other log lines surrounding that one. Maybe some SSL issues (if you're using SSL), maybe firewall issues. Hard to tell.