topic Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1 in Getting Data In

Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

abaumbusch — Wed, 23 Dec 2020 22:02:02 GMT

Greetings!

We recently upgraded our UFs throughout the environment to 8.1.0, and since the upgrade, none of the Windows based forwarders appear to be doing AD GUID/SID-to-value lookups. We have verified that evt_resolve_ad_obj = 1 is set in inputs.conf for the [WinEventLog://Security] stanza (verified with btool as well), and prior to the upgrade, the functionality was working fine. We tried installing the 8.1.1 version of the forwarder on one box as a test, but the problem persisted. Has anyone seen this or have any suggestions on what to check?

This is a multi-site clustered environment running Splunk Enterprise 8.0.7. Thanks for your help!

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

greggernigant — Wed, 13 Jan 2021 16:10:46 GMT

Have you managed to resolve this?

I have this exact same problem, unfortunately haven't found any other solution than downgrading to 8.0.x

Enabled DEBUG logging but can't find any indication of what is failing, following this article here https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/MonitorWindowseventlogdata suggest

"If you discover that SIDs are not being translated properly, you can review %SPLUNK_HOME%\var\log\splunkd.log for clues on what the problem might be. Problems with SID translations appear in the DsCrackNamesW API, which appear at the DEBUG logging level for splunkd.log, in the ExecProcessor log facility. For information on how to set the DEBUG logging level to see debug logs, see Enable debug logging in the Troubleshooting Manual."

Any suggestions appreciated!

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

abaumbusch — Wed, 13 Jan 2021 17:07:58 GMT

I opened a support case with Splunk, but research on their end is still ongoing. They thought the issue might be with Windows machines on older versions of the OS (e.g. 2012), but we are seeing this on 2016 servers as well. If I do get a solutions from them, I will make sure to post it. Thanks for your response!

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

greggernigant — Wed, 13 Jan 2021 18:10:37 GMT

Thanks!

If that helps, in my environment the issue exists on server 2016 and 2019, tried ufw 8.1.0 and the latest 8.1.1 (splunkforwarder-8.1.1-08187535c166-x64-release) without any luck.

Cheers.

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

jeff — Wed, 27 Jan 2021 23:20:33 GMT

The following has been added to known issues:

SPL-199409, SPL-199691: Windows EventLog SIDs no longer resolving after upgrade to 8.1

Splunk Devs acknowledged the bug (I work with @abaumbusch) and provided the following workarounds:

replace the splunk-winevtlog.exe binary on an 8.1.x forwarder with a copy from an 8.0.x forwarder, OR
set 'use_old_eventlog_api' to true for any WinEventLogs that need the SID/GUID translation. It was found that this issue does not affect any channel where 'use_old_eventlog_api' is enabled.

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

mmatturro — Sat, 06 Feb 2021 01:00:24 GMT

Looks like this is still an issue in 8.1.2: https://docs.splunk.com/Documentation/Forwarder/8.1.2/Forwarder/KnownIssues.

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

jeff — Sat, 06 Feb 2021 16:56:30 GMT

I was told the fix may come in 8.1.3. I can confirm adding use_old_eventlog_api = 1 to the wineventlog:security stanza in inputs conf seems to resolve the issue. I deployed this change to forwarders on test and dev systems in our environment and the sids were properly resolved.

Re: Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

pbarbuto — Thu, 17 Jun 2021 19:43:05 GMT

Do you know if using use_old_eventlog_api caused any other issues or side effects? Any issues with field extractions?