Getting Data In

[powershell://...] input working example and some pitfalls

ololdach
Builder

Hi Splunkerds,

I have struggled with powershell for a while and thought that after all the great tips I got from you, I'd repay you by posting a solution. The problem is, that the documentation for the inputs.conf section on powershell offers little to none help to actually get things right. Some things to note that the documentation seems to miss are:

- If your script returns strings, then every line of output will be a single event and it will bypass anything that you do in order to reassemble the events. Forget SHOULD_LINEMERGE and LINE_BREAKER. They are simply ignored as if the UF using the powershell input actually passes cooked events to splunk.

- Consequently, as a result of the above: If you want to pass anything more complex than a single line, you have to pass the events as powershell objects

Now for a practical and working example, let's pull the output of the certutil.exe into splunk using powershell:

First, test the command in powershell to see the raw data: 

 

 

# certutil -view -out 'NotBefore,NotAfter,SerialNumber,RequestID,CertificateTemplate,DistinguishedName'

 

 

Now, for the solution, let's take a look at inputs.conf:

 

 

[powershell://CertStore-LocalUser]
disabled = 0
script = . "$SplunkHome\etc\apps\<yourappname>\bin\Get-LocalUserCertificates.ps1"
schedule = */30 * * * *
sourcetype = windows:certs

 

 

Note the script line: It's taken me some time to get the invocation right. The docs could be clearer.

Here is the content of the Get-LocalUserCertificates.ps1 script located in the <yourappname>/bin directory as an example/template of how to take a linear (string) based input, iterate over the lines, cut them with regexes and create new objects for every Row or Entry section, assign the keys and pass them to splunk:

 

 

$certutil = "$($env:SystemRoot)\system32\certutil.exe"
$certutilOutput = Invoke-Expression "$certutil -view -out 'NotBefore,NotAfter,SerialNumber,RequestID,CertificateTemplate,DistinguishedName'"
 
$currentKey = $null
$currentValue = $null
$certutilOutput -split [environment]::NewLine | foreach {
 
    switch -regex ($_){
 
        '[Entry|Row] \d+:(.*)' {
            # New Object found
            $currentObject = New-Object psobject
            $currentValue = $null
            continue
        }
 
        '(\s{2})(?<key>[\w\s]+):\s*(?<value>.*)' {
            # Add key / value pair to Object
            if($currentObject){ 
                if( -not $matches.value){
                    $currentKey = $matches.key.Trim()
                    $currentValue = $null
                }
                else {
                    $currentObject | Add-Member -MemberType NoteProperty -Name $($matches.key.Trim()) -Value $($matches.value.trim(@("'",'"','`'))) -Force 
                }
            }
        }
 
        '^$' {
            # if it exists, pass the object to splunk
            if($currentObject){ 
                if($currentKey -and $currentValue){
                    $currentObject | Add-Member -MemberType NoteProperty -Name $($currentKey) -Value $($currentValue) -Force 
                }
                # pass the object to splunk
                $currentObject 
                # reset the object after it has been passed
                $currentObject = $null
                $currentKey = $null
                $currentValue = $null
            }
        }
        default {
            # Save Values of Multiline Parameters if a current Key exists
            if($currentObject -and $currentKey){ 
                $currentValue += $_+"`n"
            }
        }
    }
}
 
# return the last row object
if($currentKey -and $currentValue){
    $currentObject | Add-Member -MemberType NoteProperty -Name $($currentKey) -Value $($currentValue) -Force 
}
$currentObject

 

 

 Note: If your command delivers a list of PS Objects, you just iterate over that list with .Next, passing ever object that your list holds. 

Hope it helps somebody!

Happy splunkin'

Oliver

0 Karma