Getting Data In

Why is the Powershell REST API timing out?

abbam
Explorer

Hi All,

I've made a script in PowerShell which utilizes the REST API input that I have. This script is for downloading extremely large amounts of data from Splunk.

The problem that I am having is that Invoke-WebRequest is timing out and I cannot get the query to complete. The query stays connected for ~5minutes and times out.

Code:

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
[System.Net.ServicePointManager]::MaxServicePointIdleTime = 5000000
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$User = "admin"
$File = "C:\Temp\Password.txt"
$MyCredential=New-Object -TypeName System.Management.Automation.PSCredential `
 -ArgumentList $User, (Get-Content $File | ConvertTo-SecureString)
$server = 'SPLUNKSERVER'
$url = "https://${server}:8089/services/search/jobs/export"
$search = "search index=_internal earliest=-5m@m | stats count by sourcetype"
$body = @{
    search = $search
    output_mode = "csv"
    earliest_time = "-5m"
    latest_time = "now"
}
Invoke-WebRequest -Method Post -Uri $url -Timeout 3600 -Credential $MyCredential -Body $body | Set-Content C:\Users\User1\Desktop\test.csv

Thanks!

0 Karma

dstaulcu
Builder

Aside from changing the credential from file based to prompt based, the script worked fine for me against my remote instance. Of course my data set from your search was very small and therefore conditions were not present for timeout based on delay in processing large amounts of data synchronously. I've never had to write an asynchronous query before, but here is v1 of my first powershell script to do so. Let me know if that helps with your situation.

0 Karma

ansif
Motivator

Can you check the script running server to url server has communication.

Use Ping and telnet.

0 Karma

pmccomb001
New Member

This does not appear to be an answer. I have the same issue. Anyone have any thoughts?

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...