I found that the PowerShell scripts that are running for the Splunk Add-on for Microsoft Active Directory were causing some issue, not allowing the PowerShell scripts for the Splunk Add-on for SCOM to complete, so no data was getting into the index.
Steps I used to discover the issue (nothing in the logging provided a clue):
Saw that the PowerShell process on the server was consuming a LOT of memory and not releasing any.
Using Process Explorer, I found the command line that Splunk uses to launch the PowerShell scripts. It writes a temp file with the parameters and passes it to PowerShell.
I cracked open the temp file and saw that before the SCOM-related PS scripts were run, a bunch of AD-related PS scripts were executed.
I disabled the Splunk Add-on for Microsoft Active Directory.
Data from SCOM began populating the index.
However, some add-ons/apps require the Splunk Add-on for Microsoft Active Directory. When I re-enabled it, the data from SCOM stopped being written again. Can anyone elaborate why this would be the case?
... View more