Getting Data In

How to import data to Splunk via HTTP GET request?

tamduong16
Contributor

I've been looking for a way to import contents from an http get request with Splunk without success. At first, I thought I could do this by using Rest Api section that build-in Splunk. But after I give it a url to do an http get request, my search return no event. I thought this is all I have to do to get content from the page to Splunk. The documentation for this section is very confusing and I don't know where to start. At this point, I don't know if Rest Api is the answer to my question. Does anyone know of a way I could get content with an http get request in Splunk?

0 Karma
1 Solution

sjalexander
Path Finder

Did your search return anything at all? Normally, with the API you run a search and the query returns a search SID, then you use the SID to query again for the results of the search.

You would usually build your search as a set of parameters and pass it to an endpoint, and get a SID back from that. Here's an example snippet from some python code I wrote to get bucket info from a dbinspect search:

 params = "search=%7Cdbinspect%20index%3D%2A%20latest%3Dnow%20earliest%3D-99y&exec_mode=blocking"
 url = "https://" + searchhead + ":8089/services/search/jobs/"
 item = json.loads(fetchdata(url,params))
 try:
     sid = str(item['sid'])
 except:
     sys.exit(1)
 url = "https://" + searchhead + ":8089/services/search/jobs/" + sid + "/results/"
 urldata = fetchdata(url)

You can see the query is in the params variable, and I pass it to the search head (the url variable) as a query parameter (the fetchdata call). the result is returned to the item[] array which I grab the SID from (the try block: sid = str item['sid']).
Once I have the SID I build a new url var with the SID (second from last line in the example) then call it to return the result data.

View solution in original post

sjalexander
Path Finder

Did your search return anything at all? Normally, with the API you run a search and the query returns a search SID, then you use the SID to query again for the results of the search.

You would usually build your search as a set of parameters and pass it to an endpoint, and get a SID back from that. Here's an example snippet from some python code I wrote to get bucket info from a dbinspect search:

 params = "search=%7Cdbinspect%20index%3D%2A%20latest%3Dnow%20earliest%3D-99y&exec_mode=blocking"
 url = "https://" + searchhead + ":8089/services/search/jobs/"
 item = json.loads(fetchdata(url,params))
 try:
     sid = str(item['sid'])
 except:
     sys.exit(1)
 url = "https://" + searchhead + ":8089/services/search/jobs/" + sid + "/results/"
 urldata = fetchdata(url)

You can see the query is in the params variable, and I pass it to the search head (the url variable) as a query parameter (the fetchdata call). the result is returned to the item[] array which I grab the SID from (the try block: sid = str item['sid']).
Once I have the SID I build a new url var with the SID (second from last line in the example) then call it to return the result data.

Sukisen1981
Champion

in addition you can use workflow actions and alerts to GET / POST http actions as well. Remember, the REST api will poll the GET url continuously at the polling interval. Suggest read up workflow and alert actions as well, but what @sjalexander is asking is most important - if your rest api integration is not working and not fetching a SID then it could be that you have missed some configuration, are you sure you have passed the tokens / keys needed to fetch data from your GET url? Mostly all api urls will have some authentication mechanism....

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...