Getting Data In

REST API Modular Input - 401 Client Unauthorized

hpintelliflo
Explorer

I'm trying to get the REST Input to work with Google Nest API which has a space in one of the headers which I think is causing an issue. I can get other REST APIs to work on the same server. The header is the Authorization one which includes Bearer and then a key

From postman I can get to the Nest API from the server so it's not a network issue.

But splunkd.log is giving me

09-29-2017 15:10:52.452 +0000 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\rest_ta\bin\rest.py"" HTTP Request error: 401 Client Error: Unauthorized

I’ve tried putting inverted commas around it but that hasn’t fixed it. I have also tried replacing the space with %20

The inputs.conf stanza is

[rest://Nest]
auth_type = none
endpoint = https://developer-api.nest.com/
http_header_propertys = Authorization=Bearer c.hp9b{rest of key}
http_method = GET
index = nest
index_error_response_codes = 0
response_type = json
sequential_mode = 0
sourcetype = _json
streaming_request = 0
disabled = 0

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You need to add a custom response handler in rest_ta/bin/responsehandlers.py and declare this handler to be applied in your stanza setup.

This custom handler would process the response from request 1 , get the the url from the location header for request 2 and send off request 2.

View solution in original post

Damien_Dallimor
Ultra Champion

You need to add a custom response handler in rest_ta/bin/responsehandlers.py and declare this handler to be applied in your stanza setup.

This custom handler would process the response from request 1 , get the the url from the location header for request 2 and send off request 2.

hpintelliflo
Explorer

Awesome. Many thanks indeed for all your help on this one!

0 Karma

Damien_Dallimor
Ultra Champion

Some of the code examples dont have the "c." part , what happens if you remove that ?

Also try adding in the Content-Type=application/json header also as per the examples in the docs

0 Karma

hpintelliflo
Explorer

Guys - i think i have worked it out ... Nest does a 307 redirect. I suspect the module is not sending the headers on the redirect request. This Python is working

import httplib

headers = {"Authorization": "Bearer c.{INSERT KEY}"}

conn = httplib.HTTPSConnection("developer-api.nest.com")
conn.request("GET", "/", "", headers)
response = conn.getresponse()

url2 = response.getheader("location")
url2 = url2[8:-1]

conn2 = httplib.HTTPSConnection(url2)
conn2.request("GET", "/", "", headers)
response2 = conn2.getresponse()

print response2.read()
conn.close()
conn2.close()

Any thoughts?

0 Karma

hpintelliflo
Explorer

Thanks Damien. Will try. I’ve confirmed wih postman that the content type header isn’t required.

0 Karma

Damien_Dallimor
Ultra Champion

I can't really see anything else based of that inputs.conf snippet.
Maybe ensure that there is no hidden whitespace after your token and/or put in the content type header anyway as is in the Python 2.7 examples on the docs site.

0 Karma

hpintelliflo
Explorer

Guys I've tried all of these but nothing has worked ... any more ideas?

0 Karma

Damien_Dallimor
Ultra Champion

Can you compare the HTTP POST/GET data you see on the actual wire with a successful request vs a non successful request.This should tell you exactly what , if any, differences there may be at the client end ie: using ngrep, wireshark , tcpdump .. or maybe even Splunk Stream 🙂

Some examples here of using ngrep and tcpdump : https://stackoverflow.com/questions/9241391/how-to-capture-all-the-http-packets-using-tcpdump

jkat54
SplunkTrust
SplunkTrust

I’ll second this approach

0 Karma

jkat54
SplunkTrust
SplunkTrust

This reminds me of a BBQ temperature sensor I helped integrate with Splunk a year or two ago. In the end we found the temp sensors API wanted the Authorization token in the headers to be lowercase authorization. Not sure how to hack that into the rest_ta, but it's worth a shot in postman to see if you can replicate it.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...