[powershell://...] input working example and some pitfalls

[ololdach]

Hi Splunkerds,

I have struggled with powershell for a while and thought that after all the great tips I got from you, I'd repay you by posting a solution. The problem is, that the documentation for the inputs.conf section on powershell offers little to none help to actually get things right. Some things to note that the documentation seems to miss are:

- If your script returns strings, then every line of output will be a single event and it will bypass anything that you do in order to reassemble the events. Forget SHOULD_LINEMERGE and LINE_BREAKER. They are simply ignored as if the UF using the powershell input actually passes cooked events to splunk.

- Consequently, as a result of the above: If you want to pass anything more complex than a single line, you have to pass the events as powershell objects

Now for a practical and working example, let's pull the output of the certutil.exe into splunk using powershell:

First, test the command in powershell to see the raw data: 

# certutil -view -out 'NotBefore,NotAfter,SerialNumber,RequestID,CertificateTemplate,DistinguishedName'

Now, for the solution, let's take a look at inputs.conf:

[powershell://CertStore-LocalUser]
disabled = 0
script = . "$SplunkHome\etc\apps\<yourappname>\bin\Get-LocalUserCertificates.ps1"
schedule = */30 * * * *
sourcetype = windows:certs

Note the script line: It's taken me some time to get the invocation right. The docs could be clearer.

Here is the content of the Get-LocalUserCertificates.ps1 script located in the <yourappname>/bin directory as an example/template of how to take a linear (string) based input, iterate over the lines, cut them with regexes and create new objects for every Row or Entry section, assign the keys and pass them to splunk:

$certutil = "$($env:SystemRoot)\system32\certutil.exe"
$certutilOutput = Invoke-Expression "$certutil -view -out 'NotBefore,NotAfter,SerialNumber,RequestID,CertificateTemplate,DistinguishedName'"
 
$currentKey = $null
$currentValue = $null
$certutilOutput -split [environment]::NewLine | foreach {
 
    switch -regex ($_){
 
        '[Entry|Row] \d+:(.*)' {
            # New Object found
            $currentObject = New-Object psobject
            $currentValue = $null
            continue
        }
 
        '(\s{2})(?<key>[\w\s]+):\s*(?<value>.*)' {
            # Add key / value pair to Object
            if($currentObject){ 
                if( -not $matches.value){
                    $currentKey = $matches.key.Trim()
                    $currentValue = $null
                }
                else {
                    $currentObject | Add-Member -MemberType NoteProperty -Name $($matches.key.Trim()) -Value $($matches.value.trim(@("'",'"','`'))) -Force 
                }
            }
        }
 
        '^$' {
            # if it exists, pass the object to splunk
            if($currentObject){ 
                if($currentKey -and $currentValue){
                    $currentObject | Add-Member -MemberType NoteProperty -Name $($currentKey) -Value $($currentValue) -Force 
                }
                # pass the object to splunk
                $currentObject 
                # reset the object after it has been passed
                $currentObject = $null
                $currentKey = $null
                $currentValue = $null
            }
        }
        default {
            # Save Values of Multiline Parameters if a current Key exists
            if($currentObject -and $currentKey){ 
                $currentValue += $_+"`n"
            }
        }
    }
}
 
# return the last row object
if($currentKey -and $currentValue){
    $currentObject | Add-Member -MemberType NoteProperty -Name $($currentKey) -Value $($currentValue) -Force 
}
$currentObject

 Note: If your command delivers a list of PS Objects, you just iterate over that list with .Next, passing ever object that your list holds. 

Hope it helps somebody!

Happy splunkin'

Oliver

[lithium43]

Thanks for your response and the pieces are starting to fall into place.  It still seems confusing that I'm seeing [script://....] that seems to call PowerShell as well as [powershell://...] also calling PowerShell with one using schedule and the other using interval.  Really looks messy which might explain why .bat and .py scipts seem to work so much better (especially on Unix systems) but this could be my OCD kicking in.

[PickleRick]

These are two separate mechanisms. Powershell has some features that script input doesn't (most important being the ability to receive powershell objects, not just text).

[lithium43]

This may be a noob question but what is the first line of the inputs.conf entry defining?  Does everything within the brackets, [powershell://CertStore-LocalUser], provide function or is this a commented area?

[PickleRick]

It's a stanza defining a type of input (powershell in this case) and a unique name for it. In some cases (for example the monitor inputs) input names also have functional meaning (monitor input name points to monitored files or directories on disk).