Getting Data In

Why is the Powershell REST API timing out?

abbam
Explorer

Hi All,

I've made a script in PowerShell which utilizes the REST API input that I have. This script is for downloading extremely large amounts of data from Splunk.

The problem that I am having is that Invoke-WebRequest is timing out and I cannot get the query to complete. The query stays connected for ~5minutes and times out.

Code:

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
[System.Net.ServicePointManager]::MaxServicePointIdleTime = 5000000
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$User = "admin"
$File = "C:\Temp\Password.txt"
$MyCredential=New-Object -TypeName System.Management.Automation.PSCredential `
 -ArgumentList $User, (Get-Content $File | ConvertTo-SecureString)
$server = 'SPLUNKSERVER'
$url = "https://${server}:8089/services/search/jobs/export"
$search = "search index=_internal earliest=-5m@m | stats count by sourcetype"
$body = @{
    search = $search
    output_mode = "csv"
    earliest_time = "-5m"
    latest_time = "now"
}
Invoke-WebRequest -Method Post -Uri $url -Timeout 3600 -Credential $MyCredential -Body $body | Set-Content C:\Users\User1\Desktop\test.csv

Thanks!

0 Karma

dstaulcu
Builder

Aside from changing the credential from file based to prompt based, the script worked fine for me against my remote instance. Of course my data set from your search was very small and therefore conditions were not present for timeout based on delay in processing large amounts of data synchronously. I've never had to write an asynchronous query before, but here is v1 of my first powershell script to do so. Let me know if that helps with your situation.

0 Karma

ansif
Motivator

Can you check the script running server to url server has communication.

Use Ping and telnet.

0 Karma

pmccomb001
New Member

This does not appear to be an answer. I have the same issue. Anyone have any thoughts?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...