TA-sos_win: Powershell Script Error

[rturk]

Hello Splunkers and/or Powershell Gurus!

I'm getting a bunch of errors when using the Splunk-on-Splunk TA for the collection of diagnostic data. I have enabled Powershell script execution, and I'm using the app as packaged deployed by a Cluster Master to an indexer.

Each of the lines below is prefixed with:

10-24-2013 08:30:06.594 +1100 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\etc\slave-apps\TA-sos_win\bin\sospowershell.cmd" ps_sos.ps1

The reported error:

Error formatting a string: Index (zero based) must be greater than or equal to 
zero and less than the size of the argument list..
At C:\Program 
Files\Splunk\etc\slave-apps\TA-sos_win\bin\powershell\ps_sos.ps1:17 char:5
 +     $CMDLINE = "{0}" -f ($CMDLINE -replace '"',"")
 +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     + CategoryInfo          : InvalidOperation: ({0}:String) [], RuntimeExcept 
    ion
     + FullyQualifiedErrorId : FormatError

Any & all help greatly appreciated 🙂

[hexx · Accepted solution]

UPDATE: The latest version of the S.o.S app and of its add-on for Windows have now been released and fix this particular issue.

---

Please upgrade your TA to version 2.3.2. We have made improvements in that version to address this particular error. From the release notes:

• [SUP-691] Fixed an issue where the ps_sos.ps1 scripted input produced format errors when trying to set $CMDLINE.

Do note, however, that to the best of our knowledge these errors typically bubble up when the execution of the script fails because the Get-Process cmdlet somehow fails to return output. We are still tracking down the root cause for that particular problem, which will cause ps_sos.ps1 to intermittently fail to return output.

The bug fixes in version 2.3.2 should ensure that ps_sos.ps1 will handle these errors more gracefully.

[rturk]

Hi Hexx - Sorry for the delay in getting back to you. Unfortunately the upgrade didn't help and we're still seeing a LOT of the above errors. Being that we know what the cause is and it's a known bug, I'll mark this as answered, but I look forward to the next release where (hopefully) it'll be fixed.

Thanks again 🙂

[rturk]

version = 2.3.1

[hexx]

What version of the S.o.S TA for Windows are you running?

[rturk]

It's intermittent in nature. e.g.

index=_internal ERROR greater | rex "2013 (?[^\s]+)\s" | stats values(time_of_error)

00:30:06.232
00:42:06.072
00:42:16.605
01:30:08.017
01:30:08.235
01:50:02.871
01:55:03.419
02:10:02.950
02:10:03.575
02:15:03.380
...

[hexx]

What version of the S.o.S TA for Windows are you running?

Also: Are these errors happening on every 5s-interval run of the scripted input or are they happening intermittently? If the latter, what seems to be the frequency?