I found that the PowerShell scripts that are running for the Splunk Add-on for Microsoft Active Directory were causing some issue, not allowing the PowerShell scripts for the Splunk Add-on for SCOM to complete, so no data was getting into the index.
Steps I used to discover the issue (nothing in the logging provided a clue):
Saw that the PowerShell process on the server was consuming a LOT of memory and not releasing any.
Using Process Explorer, I found the command line that Splunk uses to launch the PowerShell scripts. It writes a temp file with the parameters and passes it to PowerShell.
I cracked open the temp file and saw that before the SCOM-related PS scripts were run, a bunch of AD-related PS scripts were executed.
I disabled the Splunk Add-on for Microsoft Active Directory.
Data from SCOM began populating the index.
However, some add-ons/apps require the Splunk Add-on for Microsoft Active Directory. When I re-enabled it, the data from SCOM stopped being written again. Can anyone elaborate why this would be the case?
... View more
I have setup the Splunk Add-on for Microsoft System Center Operations Manager (SCOM) and have successfully setup a performance input from SCOM and can see in the ta_scom.log that the PowerShell scripts are getting the objects. However, nothing is written to the index. In fact, I don't see the add-on sourcetypes at all ("microsoft:scom:*")
I've exhausted all the log files I can think of to look in for clues as to why the data never makes it into the index, but so far have come up empty. Any ideas as to what I'm missing or where else I can look to troubleshoot the issue?
Thanks in advance!
... View more