Getting Data In

Why is the Powershell REST API timing out?

abbam
Explorer

Hi All,

I've made a script in PowerShell which utilizes the REST API input that I have. This script is for downloading extremely large amounts of data from Splunk.

The problem that I am having is that Invoke-WebRequest is timing out and I cannot get the query to complete. The query stays connected for ~5minutes and times out.

Code:

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
[System.Net.ServicePointManager]::MaxServicePointIdleTime = 5000000
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$User = "admin"
$File = "C:\Temp\Password.txt"
$MyCredential=New-Object -TypeName System.Management.Automation.PSCredential `
 -ArgumentList $User, (Get-Content $File | ConvertTo-SecureString)
$server = 'SPLUNKSERVER'
$url = "https://${server}:8089/services/search/jobs/export"
$search = "search index=_internal earliest=-5m@m | stats count by sourcetype"
$body = @{
    search = $search
    output_mode = "csv"
    earliest_time = "-5m"
    latest_time = "now"
}
Invoke-WebRequest -Method Post -Uri $url -Timeout 3600 -Credential $MyCredential -Body $body | Set-Content C:\Users\User1\Desktop\test.csv

Thanks!

0 Karma

dstaulcu
Builder

Aside from changing the credential from file based to prompt based, the script worked fine for me against my remote instance. Of course my data set from your search was very small and therefore conditions were not present for timeout based on delay in processing large amounts of data synchronously. I've never had to write an asynchronous query before, but here is v1 of my first powershell script to do so. Let me know if that helps with your situation.

0 Karma

ansif
Motivator

Can you check the script running server to url server has communication.

Use Ping and telnet.

0 Karma

pmccomb001
New Member

This does not appear to be an answer. I have the same issue. Anyone have any thoughts?

0 Karma
Get Updates on the Splunk Community!

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...