Splunk Search

Correct Search string for search using the Splunk API

bsonposh
Communicator

I want to be able to do a search like "UserName=Bleh sourcetype=ns_log" but it doesn't seem to work. Does the API use a different syntax than the GUI?

Tags (2)
1 Solution

bsonposh
Communicator

I found this link which is helpful:

This answer was two fold.

1) Finding the write string to send. 2) Figuring out how to send the correct POST string via .NET/Powershell.

For one I used: http://www.splunk.com/base/Documentation/latest/SearchReference/Search

For two I add to build the string and use System.Web.HttpUtility.UrlEncode and then convert the string to a byte array.

Here is the sample code (Powershell)

function New-SplunkSearchJob
{

    [Cmdletbinding()]
    Param(

        [Parameter()]
        [String]$Server = $Splunk_Server,

        [Parameter()]
        [int]$Port = $Splunk_Port,

        # Search parameters support in POST
        # http://www.splunk.com/base/Documentation/latest/Developer/RESTSearch#POST

        [Parameter()]           # search
        [String]$Search = "search *",

        [Parameter()]           # required_field_list (comma separated list)
        [String]$RequireFields,

        [Parameter()]           # earliest_time
        [String]$StartDate,

        [Parameter()]           # latest_time
        [String]$EndDate,

        [Parameter()]           # id
        [String]$ID,

        [Parameter()]           # max_count = int
        [int]$MaxCount,

        [Parameter()]           # max_time = int
        [int]$MaxTime,

        [Parameter()]
        [System.Management.Automation.PSCredential]$Creds = $Splunk_Credentials

        # Plan to implement

        #[Parameter()]           # exec_mode = blocking | oneshot | normal (only supporting oneshot/normal)
        #[Switch]$Wait,      

        #[Parameter()]           # 'search_mode: normal | realtime'
        #[Switch]$Realtime,

    )

    function Get-Bytes($String)
    {
        [Byte[]]$byteArray = [System.Text.Encoding]::UTF8.GetBytes($String)
        $byteArray
    }

    $URL = "https://${Server}:${Port}/services/search/jobs" 

    Write-Verbose " [New-SplunkSearchJob] :: URL = $URL"

    $Request = [System.Net.WebRequest]::Create($url)
    $Request.Credentials = $Creds
    $Request.Method ="POST"
    $Request.ContentType = "application/x-www-form-urlencoded"
    $RequestStream = $Request.GetRequestStream()

    Write-Verbose " [New-SplunkSearchJob] :: Creating POST message"

    Write-Verbose " [New-SplunkSearchJob] :: Adding Search string [search=$Search] to POST message"
    #[string]$PostString = "search=$Search"
    [string]$PostString = "search={0}" -f [System.Web.HttpUtility]::UrlEncode($search)

    if($StartDate)
    {
        $PostString += "&earliest_time={0}" -f [System.Web.HttpUtility]::UrlEncode($StartDate)
    }

    if($EndDate)
    {
        $PostString += "&latest_time={0}" -f [System.Web.HttpUtility]::UrlEncode($EndDate)
    }

    if($MaxCount)
    {
        $PostString += "&max_count={0}" -f [System.Web.HttpUtility]::UrlEncode($MaxCount)
    }

    if($MaxTime)
    {
        $PostString += "&max_time={0}" -f [System.Web.HttpUtility]::UrlEncode($MaxTime)
    }


    Write-Verbose " [New-SplunkSearchJob] :: `$PostString = $PostString"

    Write-Verbose " [New-SplunkSearchJob] :: Converting POST message to Byte Array"
    $Bytes = Get-Bytes $PostString

    Write-Verbose " [New-SplunkSearchJob] :: Sending POST message"
    $RequestStream.Write($Bytes,0,$Bytes.length)

    Write-Verbose " [New-SplunkSearchJob] :: Closing POST stream"
    $RequestStream.Close()

    Write-Verbose " [New-SplunkSearchJob] :: Getting Response from POST"
    $Response = $Request.GetResponse()
    $Reader = new-object System.IO.StreamReader($Response.GetResponseStream())

    [XML]$Results = $Reader.ReadToEnd()

    $SID = $Results.Response.sid
    Write-Verbose " [New-SplunkSearchJob] :: ID = $SID"

    if($SID)
    {
        $cont = $true
        Write-Host "Please wait. It could take a bit..." -NoNewline
        while($cont)
        {
            $Job = Get-SplunkSearchJob -Filter $SID | ?{$_.isDone -eq 1}
            if($Job.ID)
            {
                $Job | Get-SplunkSearchJobResult 
                $cont = $false
                Write-Host
                Write-Host "Search complete"
                continue
            }
            sleep 1
        }
    }
}    # New-SplunkSearchJob

View solution in original post

bsonposh
Communicator

I found this link which is helpful:

This answer was two fold.

1) Finding the write string to send. 2) Figuring out how to send the correct POST string via .NET/Powershell.

For one I used: http://www.splunk.com/base/Documentation/latest/SearchReference/Search

For two I add to build the string and use System.Web.HttpUtility.UrlEncode and then convert the string to a byte array.

Here is the sample code (Powershell)

function New-SplunkSearchJob
{

    [Cmdletbinding()]
    Param(

        [Parameter()]
        [String]$Server = $Splunk_Server,

        [Parameter()]
        [int]$Port = $Splunk_Port,

        # Search parameters support in POST
        # http://www.splunk.com/base/Documentation/latest/Developer/RESTSearch#POST

        [Parameter()]           # search
        [String]$Search = "search *",

        [Parameter()]           # required_field_list (comma separated list)
        [String]$RequireFields,

        [Parameter()]           # earliest_time
        [String]$StartDate,

        [Parameter()]           # latest_time
        [String]$EndDate,

        [Parameter()]           # id
        [String]$ID,

        [Parameter()]           # max_count = int
        [int]$MaxCount,

        [Parameter()]           # max_time = int
        [int]$MaxTime,

        [Parameter()]
        [System.Management.Automation.PSCredential]$Creds = $Splunk_Credentials

        # Plan to implement

        #[Parameter()]           # exec_mode = blocking | oneshot | normal (only supporting oneshot/normal)
        #[Switch]$Wait,      

        #[Parameter()]           # 'search_mode: normal | realtime'
        #[Switch]$Realtime,

    )

    function Get-Bytes($String)
    {
        [Byte[]]$byteArray = [System.Text.Encoding]::UTF8.GetBytes($String)
        $byteArray
    }

    $URL = "https://${Server}:${Port}/services/search/jobs" 

    Write-Verbose " [New-SplunkSearchJob] :: URL = $URL"

    $Request = [System.Net.WebRequest]::Create($url)
    $Request.Credentials = $Creds
    $Request.Method ="POST"
    $Request.ContentType = "application/x-www-form-urlencoded"
    $RequestStream = $Request.GetRequestStream()

    Write-Verbose " [New-SplunkSearchJob] :: Creating POST message"

    Write-Verbose " [New-SplunkSearchJob] :: Adding Search string [search=$Search] to POST message"
    #[string]$PostString = "search=$Search"
    [string]$PostString = "search={0}" -f [System.Web.HttpUtility]::UrlEncode($search)

    if($StartDate)
    {
        $PostString += "&earliest_time={0}" -f [System.Web.HttpUtility]::UrlEncode($StartDate)
    }

    if($EndDate)
    {
        $PostString += "&latest_time={0}" -f [System.Web.HttpUtility]::UrlEncode($EndDate)
    }

    if($MaxCount)
    {
        $PostString += "&max_count={0}" -f [System.Web.HttpUtility]::UrlEncode($MaxCount)
    }

    if($MaxTime)
    {
        $PostString += "&max_time={0}" -f [System.Web.HttpUtility]::UrlEncode($MaxTime)
    }


    Write-Verbose " [New-SplunkSearchJob] :: `$PostString = $PostString"

    Write-Verbose " [New-SplunkSearchJob] :: Converting POST message to Byte Array"
    $Bytes = Get-Bytes $PostString

    Write-Verbose " [New-SplunkSearchJob] :: Sending POST message"
    $RequestStream.Write($Bytes,0,$Bytes.length)

    Write-Verbose " [New-SplunkSearchJob] :: Closing POST stream"
    $RequestStream.Close()

    Write-Verbose " [New-SplunkSearchJob] :: Getting Response from POST"
    $Response = $Request.GetResponse()
    $Reader = new-object System.IO.StreamReader($Response.GetResponseStream())

    [XML]$Results = $Reader.ReadToEnd()

    $SID = $Results.Response.sid
    Write-Verbose " [New-SplunkSearchJob] :: ID = $SID"

    if($SID)
    {
        $cont = $true
        Write-Host "Please wait. It could take a bit..." -NoNewline
        while($cont)
        {
            $Job = Get-SplunkSearchJob -Filter $SID | ?{$_.isDone -eq 1}
            if($Job.ID)
            {
                $Job | Get-SplunkSearchJobResult 
                $cont = $false
                Write-Host
                Write-Host "Search complete"
                continue
            }
            sleep 1
        }
    }
}    # New-SplunkSearchJob
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Kubernetes Alerts That Work Out of the Box with AutoDetect detectors in Splunk ...

A deployment rolls out slowly, one pod sits not ready for a few minutes, and whoever is on call gets an alert ...

Observability at Your Fingertips Directly From Splunk Platform—for Free

At .conf, we announced our roadmap to bring observability foundations to Splunk Platform. Bringing metrics and ...