We upgraded to Server2012R2 Domain controllers and so I changed the addon TA for Windows Infrastructure from TA-DomainController-NT6 to TA-DomainController-2012R2, and I get the following Stanza errors when starting the splunk forwarder:
(Replaced Backslashes with Forwardslashes for pasting here)
Checking prerequisites...
Checking mgmt port [8089]: open
Checking conf files for problems...
Invalid key in stanza [powershell://Replication-Stats] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 75: script (value: & "$SplunkHome/etc/apps/TA-DomainController-2012R2/bin/Invoke-MonitoredScript.ps1" -Command "./replication-stats.ps1")
Invalid key in stanza [powershell://Replication-Stats] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 76: schedule (value: 30 */5 * ? * *)
Invalid key in stanza [powershell://AD-Health] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 86: script (value: & "$SplunkHome/etc/apps/TA-DomainController-2012R2/bin/Inv
oke-MonitoredScript.ps1" -Command "./ad-health.ps1")
Invalid key in stanza [powershell://AD-Health] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 87: schedule (value: 0 */5 * ? * *)
Invalid key in stanza [powershell://Siteinfo] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 97: script (value: & "$SplunkHome/etc/apps/TA-DomainController-2012R2/bin/Invo
ke-MonitoredScript.ps1" -Command "./siteinfo.ps1")
Invalid key in stanza [powershell://Siteinfo] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 98: schedule (value: 0 15 * ? * *)
Your indexes and inputs configurations are not internally consistent. For more information, run 'splunk btool check --debug'
Done
So, pretty much... the powershell scripts won't run anymore.
Just hazarding a guess: did you also deploy the Splunk App for Microsoft Powershell? The Exchange App Documentation says it's a requirement for TA-DomainController-2012R2.
I fixed the formatting for you. When you have a bit of code or output, select the text and click the "101010" button. That will preserve linebreaks and make things easier to read.
Just hazarding a guess: did you also deploy the Splunk App for Microsoft Powershell? The Exchange App Documentation says it's a requirement for TA-DomainController-2012R2.
Yes, The Powershell App was a requirement prior to updating from TA-DomainController-NT6 to TA-DomainController-2012R2. The only thing I did was upgrade the addon to the Infrastructure App, since we upgraded our domain controllers. The stanzas to launch Powershell scripts are completely different between the two. One Stanza starts with "Script", the Other Starts with "Powershell".
TA-DomainController-NT6:
[script://.\bin\runpowershell.cmd dns-zoneinfo.ps1]
source=Powershell
sourcetype=MSAD:NT6:DNS-Zone-Information
index=msad
interval=3600
disabled=false
TA-DomainController-2012R2
[powershell://Replication-Stats]
script = & "$SplunkHome\etc\apps\TA-DomainController-2012R2\bin\Invoke-MonitoredScript.ps1" -Command ".\replication-stats.ps1"
schedule = 30 */5 * ? * *
index = msad
source = Powershell
sourcetype=MSAD:NT6:Replication
disabled=false
I think this is the right root cause--something to do with the PowerShell modinput. That error message is saying that splunkd doesn't know what to do with the powershell lines. I suspect that the modular input is not installed correctly. On the same machine which is running the TA-DomainController-2012R2 app (your DC, presumably), be sure that the PowerShell modinput is also in the etc/apps folder. Double-check permissions?
You could also try running (at an elevated prompt):
splunk.exe btool check --debug
You were right. I didn't have the Splunk App for Microsoft Powershell installed. I could have sworn that I did, though. Thanks!