Getting Data In

How to import data to Splunk via HTTP GET request?

tamduong16
Contributor

I've been looking for a way to import contents from an http get request with Splunk without success. At first, I thought I could do this by using Rest Api section that build-in Splunk. But after I give it a url to do an http get request, my search return no event. I thought this is all I have to do to get content from the page to Splunk. The documentation for this section is very confusing and I don't know where to start. At this point, I don't know if Rest Api is the answer to my question. Does anyone know of a way I could get content with an http get request in Splunk?

0 Karma
1 Solution

sjalexander
Path Finder

Did your search return anything at all? Normally, with the API you run a search and the query returns a search SID, then you use the SID to query again for the results of the search.

You would usually build your search as a set of parameters and pass it to an endpoint, and get a SID back from that. Here's an example snippet from some python code I wrote to get bucket info from a dbinspect search:

 params = "search=%7Cdbinspect%20index%3D%2A%20latest%3Dnow%20earliest%3D-99y&exec_mode=blocking"
 url = "https://" + searchhead + ":8089/services/search/jobs/"
 item = json.loads(fetchdata(url,params))
 try:
     sid = str(item['sid'])
 except:
     sys.exit(1)
 url = "https://" + searchhead + ":8089/services/search/jobs/" + sid + "/results/"
 urldata = fetchdata(url)

You can see the query is in the params variable, and I pass it to the search head (the url variable) as a query parameter (the fetchdata call). the result is returned to the item[] array which I grab the SID from (the try block: sid = str item['sid']).
Once I have the SID I build a new url var with the SID (second from last line in the example) then call it to return the result data.

View solution in original post

sjalexander
Path Finder

Did your search return anything at all? Normally, with the API you run a search and the query returns a search SID, then you use the SID to query again for the results of the search.

You would usually build your search as a set of parameters and pass it to an endpoint, and get a SID back from that. Here's an example snippet from some python code I wrote to get bucket info from a dbinspect search:

 params = "search=%7Cdbinspect%20index%3D%2A%20latest%3Dnow%20earliest%3D-99y&exec_mode=blocking"
 url = "https://" + searchhead + ":8089/services/search/jobs/"
 item = json.loads(fetchdata(url,params))
 try:
     sid = str(item['sid'])
 except:
     sys.exit(1)
 url = "https://" + searchhead + ":8089/services/search/jobs/" + sid + "/results/"
 urldata = fetchdata(url)

You can see the query is in the params variable, and I pass it to the search head (the url variable) as a query parameter (the fetchdata call). the result is returned to the item[] array which I grab the SID from (the try block: sid = str item['sid']).
Once I have the SID I build a new url var with the SID (second from last line in the example) then call it to return the result data.

Sukisen1981
Champion

in addition you can use workflow actions and alerts to GET / POST http actions as well. Remember, the REST api will poll the GET url continuously at the polling interval. Suggest read up workflow and alert actions as well, but what @sjalexander is asking is most important - if your rest api integration is not working and not fetching a SID then it could be that you have missed some configuration, are you sure you have passed the tokens / keys needed to fetch data from your GET url? Mostly all api urls will have some authentication mechanism....

Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...