Getting Data In

Windows Universal Forwarders no longer performing AD GUID/SID-to-value lookups after upgrade to 8.1

abaumbusch
Explorer

Greetings!

We recently upgraded our UFs throughout the environment to 8.1.0, and since the upgrade, none of the Windows based forwarders appear to be doing AD GUID/SID-to-value lookups. We have verified that  evt_resolve_ad_obj = 1 is set in inputs.conf for the [WinEventLog://Security] stanza (verified with btool as well), and prior to the upgrade, the functionality was working fine. We tried installing the 8.1.1 version of the forwarder on one box as a test, but the problem persisted. Has anyone seen this or have any suggestions on what to check?

This is a multi-site clustered environment running Splunk Enterprise 8.0.7. Thanks for your help!

Labels (3)
0 Karma
1 Solution

jeff
Contributor

I was told the fix may come in 8.1.3. I can confirm adding  use_old_eventlog_api = 1 to the wineventlog:security stanza in inputs conf seems to resolve the issue. I deployed this change to forwarders on test and dev systems in our environment and the sids were properly resolved.

View solution in original post

mmatturro
New Member
0 Karma

jeff
Contributor

I was told the fix may come in 8.1.3. I can confirm adding  use_old_eventlog_api = 1 to the wineventlog:security stanza in inputs conf seems to resolve the issue. I deployed this change to forwarders on test and dev systems in our environment and the sids were properly resolved.

pbarbuto
Path Finder

Do you know if using use_old_eventlog_api caused any other issues or side effects? Any issues with field extractions?

0 Karma

jeff
Contributor

The following has been added to known issues:

SPL-199409, SPL-199691: Windows EventLog SIDs no longer resolving after upgrade to 8.1 

Splunk Devs acknowledged the bug (I work with @abaumbusch) and provided the following workarounds:

  • replace the splunk-winevtlog.exe binary on an 8.1.x forwarder with a copy from an 8.0.x forwarder, OR
  • set 'use_old_eventlog_api' to true for any WinEventLogs that need the SID/GUID translation. It was found that this issue does not affect any channel where 'use_old_eventlog_api' is enabled.

abaumbusch
Explorer

I opened a support case with Splunk, but research on their end is still ongoing. They thought the issue might be with Windows machines on older versions of the OS (e.g. 2012), but we are seeing this on 2016 servers as well. If I do get a solutions from them, I will make sure to post it. Thanks for your response!

0 Karma

greggernigant
Explorer

Thanks!

If that helps, in my environment the issue exists on server 2016 and 2019, tried ufw 8.1.0 and the latest 8.1.1 (splunkforwarder-8.1.1-08187535c166-x64-release) without any luck.

Cheers.

0 Karma

greggernigant
Explorer

Have you managed to resolve this?

I have this exact same problem, unfortunately haven't found any other solution than downgrading to 8.0.x

Enabled DEBUG logging but can't find any indication of what is failing, following this article here https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/MonitorWindowseventlogdata suggest 

"If you discover that SIDs are not being translated properly, you can review %SPLUNK_HOME%\var\log\splunkd.log for clues on what the problem might be. Problems with SID translations appear in the DsCrackNamesW API, which appear at the DEBUG logging level for splunkd.log, in the ExecProcessor log facility. For information on how to set the DEBUG logging level to see debug logs, see Enable debug logging in the Troubleshooting Manual." 

Any suggestions appreciated!

Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...