Getting Data In

PowerShell sample for HTTP Event Collector

gmartins_splunk
Splunk Employee
Splunk Employee

I've been studying and creating several pieces of code to take advantage of the wonders of the HTTP Event Collector and noticed noone published a PowerShell sample, then since I created one I decided to share it with you all:

$response = ""
$formatteddate = "{0:MM/dd/yyyy hh:mm:sstt zzz}" -f (Get-Date)
$arraySeverity = 'INFO','WARN','ERROR'
$severity = $arraySeverity[(Get-Random -Maximum ([array]$arraySeverity).count)]

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", 'Splunk 653C164D-0AFB-4DFC-ADE0-D9084B03490F')

$body = '{
        "host":"' + $env:computername + '",
        "sourcetype":"testevents",
        "source":"Geoff''s PowerShell Script",
        "event":{
            "message":"Something Happened on host ' + $env:computername + '",
            "severity":"' + $severity + '",
            "user": "'+ $env:username + '",
            "date":"' + $formatteddate + '"
            }
        }'

$splunkserver = "http://yoursplunkserver.com:8088/services/collector/event"
$response = Invoke-RestMethod -Uri $splunkserver -Method Post -Headers $headers -Body $body
"Code:'" + $response.code + "' text:'"+ $response.text + "'" 

The key to the communication is the "Invoke-RestMethod" command, which is capable of performing the http call necessary to communicate with Splunk. Authentication headers for this method must be passed as a dictionary object so I created a collection of one to define the Authorization token. Everything else is pretty much straightforward, just the same as using curl.

Since PowerShell is JSON-friendly, you can capture the JSON response directly and use as variables like I did on the last line and perform some error treatment if necessary. If everything is right, this script should return something like "Code:'0' text:'Success'"

Thanks to Glenn Block for posting useful articles on the HTTP Event Collector and providing invaluable help on this and C# SDK.

1 Solution

halr9000
Motivator

Nice one. I have some comments to simplify the example:

  • Try Get-Date's -format parameter, it's easier than doing using a string formatter as you did in line 2. It takes the same .NET datetime format strings.
  • Speaking of format strings, try this one: "o". Yup, one character, and the output is very easy for Splunk to parse.

    PS C:\Users\hal> Get-Date -format o
    2016-02-25T10:52:20.8004451-08:00

  • Line 4: nothing wrong with it, but casting to [array] isn't necessary, nor are those parens. This would work fine: ... -Max $arraySeverity.Count

  • My favorite tip here, you don't need to make .NET dictionaries! A regular PowerShell hashtable will be coerced correctly. Example:

    $header = @{Authorization = "Splunk $token"}

  • Last but not least, hashtables are also great for making your JSON input, along with the ConvertTo-Json cmdlet. This eliminates the need for some of the syntax bits you used. (Although if you do this as a one-liner, you will need to terminate each assignment with a semi-colon. I left them out here because CRLF does the trick.)
    $event = @{
    host = $env:COMPUTERNAME
    sourcetype = "testevents"
    event = @{
    message = "something happened"
    severity = $severity
    }
    } | ConvertTo-Json
    Output:

    {
    "host": "DESKTOP-S7US6VG",
    "event": {
    "message": "something happened",
    "severity": null
    },
    "sourcetype": "testevents"
    }

Other notes:

  • PowerShell (technically, the underlying .NET methods) hates self-signed certificates. Best to test with SSL off on your Splunk event collector settings, and if using SSL in production, use a real certificate. For much more detail and workarounds, see @jaykul post/code about it: https://github.com/Jaykul/Tunable-SSL-Validator

View solution in original post

halr9000
Motivator

Nice one. I have some comments to simplify the example:

  • Try Get-Date's -format parameter, it's easier than doing using a string formatter as you did in line 2. It takes the same .NET datetime format strings.
  • Speaking of format strings, try this one: "o". Yup, one character, and the output is very easy for Splunk to parse.

    PS C:\Users\hal> Get-Date -format o
    2016-02-25T10:52:20.8004451-08:00

  • Line 4: nothing wrong with it, but casting to [array] isn't necessary, nor are those parens. This would work fine: ... -Max $arraySeverity.Count

  • My favorite tip here, you don't need to make .NET dictionaries! A regular PowerShell hashtable will be coerced correctly. Example:

    $header = @{Authorization = "Splunk $token"}

  • Last but not least, hashtables are also great for making your JSON input, along with the ConvertTo-Json cmdlet. This eliminates the need for some of the syntax bits you used. (Although if you do this as a one-liner, you will need to terminate each assignment with a semi-colon. I left them out here because CRLF does the trick.)
    $event = @{
    host = $env:COMPUTERNAME
    sourcetype = "testevents"
    event = @{
    message = "something happened"
    severity = $severity
    }
    } | ConvertTo-Json
    Output:

    {
    "host": "DESKTOP-S7US6VG",
    "event": {
    "message": "something happened",
    "severity": null
    },
    "sourcetype": "testevents"
    }

Other notes:

  • PowerShell (technically, the underlying .NET methods) hates self-signed certificates. Best to test with SSL off on your Splunk event collector settings, and if using SSL in production, use a real certificate. For much more detail and workarounds, see @jaykul post/code about it: https://github.com/Jaykul/Tunable-SSL-Validator

gmartins_splunk
Splunk Employee
Splunk Employee

Great remarks and modifications to my humble original example. great job @halr9000 !

0 Karma

halr9000
Motivator

You get all the credit for doing it first, and inspiring me to add some polish. 🙂

0 Karma

halr9000
Motivator

Here's what I came up with for a very basic example taking the above into account:

$token = "C3AD6A0B-2499-4070-AA51-765640DB9107"
$server = "192.168.99.101"
$port = 32771
$url = "http://${server}:$port/services/collector/event"
$header = @{Authorization = "Splunk $token"}
$event = @{event = "hello world"} | ConvertTo-Json
Invoke-RestMethod -Method Post -Uri $url -Headers $header -Body $event

Note: the reason there are curly braces around "server" above is to prevent the following colon from being picked up as a scope modifier. Otherwise it would parse to $server:32271 which is valid syntax, but would result in an empty result.

And the one-liner to compete with curl!

PS > irm -Method Post -Uri "http://192.168.99.101:32771/services/collector/event" -Headers @{Authorization = "Splunk C3AD6A0B-2499-4070-AA51-765640DB9107"} -Body '{"event": "hello world"}'

text    code
----    ----
Success    0

Arpit_S
Path Finder

Hi @halr9000 ,

I was able to send test events using the below command few days back.

irm -Method Post -Uri "https://URL.com/services/collector/event" -Headers @{Authorization = "Splunk token"} -Body '{"event": "test1 "}'

But when I tried sending a test event today it gave me an error.

irm : The underlying connection was closed: An unexpected error occurred on a send.
At line:1 char:1
+ irm -Method Post -Uri "https://URL.com/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebE
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Any idea what could be causing this?

Thanks.

0 Karma

sara_mason
Engager

Often that error ties to TLS settings. Force TLS1.2 with this command :

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

0 Karma

efavreau
Motivator

Token expired? Try getting more information from your error. Look here: https://stackoverflow.com/questions/38419325/catching-full-exception-message

###

If this reply helps you, an upvote would be appreciated.
0 Karma

Arpit_S
Path Finder

The is working fine when sending data using curl command from Linux OS.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...