topic Re: REST API Modular Input - 401 Client Unauthorized in Getting Data In

REST API Modular Input - 401 Client Unauthorized

hpintelliflo — Tue, 29 Sep 2020 16:03:43 GMT

I'm trying to get the REST Input to work with Google Nest API which has a space in one of the headers which I think is causing an issue. I can get other REST APIs to work on the same server. The header is the Authorization one which includes Bearer and then a key

From postman I can get to the Nest API from the server so it's not a network issue.

But splunkd.log is giving me

09-29-2017 15:10:52.452 +0000 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\rest_ta\bin\rest.py"" HTTP Request error: 401 Client Error: Unauthorized

I’ve tried putting inverted commas around it but that hasn’t fixed it. I have also tried replacing the space with %20

The inputs.conf stanza is

[rest://Nest]
auth_type = none
endpoint = https://developer-api.nest.com/
http_header_propertys = Authorization=Bearer c.hp9b{rest of key}
http_method = GET
index = nest
index_error_response_codes = 0
response_type = json
sequential_mode = 0
sourcetype = _json
streaming_request = 0
disabled = 0

Re: REST API Modular Input - 401 Client Unauthorized

Damien_Dallimor — Sun, 01 Oct 2017 01:11:27 GMT

Some of the code examples dont have the "c." part , what happens if you remove that ?

Also try adding in the Content-Type=application/json header also as per the examples in the docs

Re: REST API Modular Input - 401 Client Unauthorized

hpintelliflo — Sun, 01 Oct 2017 01:17:08 GMT

Thanks Damien. Will try. I’ve confirmed wih postman that the content type header isn’t required.

Re: REST API Modular Input - 401 Client Unauthorized

Damien_Dallimor — Sun, 01 Oct 2017 01:34:40 GMT

I can't really see anything else based of that inputs.conf snippet.
Maybe ensure that there is no hidden whitespace after your token and/or put in the content type header anyway as is in the Python 2.7 examples on the docs site.

Re: REST API Modular Input - 401 Client Unauthorized

jkat54 — Sun, 01 Oct 2017 12:22:51 GMT

This reminds me of a BBQ temperature sensor I helped integrate with Splunk a year or two ago. In the end we found the temp sensors API wanted the Authorization token in the headers to be lowercase authorization. Not sure how to hack that into the rest_ta, but it's worth a shot in postman to see if you can replicate it.

Re: REST API Modular Input - 401 Client Unauthorized

hpintelliflo — Mon, 09 Oct 2017 22:47:51 GMT

Guys I've tried all of these but nothing has worked ... any more ideas?

Re: REST API Modular Input - 401 Client Unauthorized

Damien_Dallimor — Mon, 09 Oct 2017 23:05:11 GMT

Can you compare the HTTP POST/GET data you see on the actual wire with a successful request vs a non successful request.This should tell you exactly what , if any, differences there may be at the client end ie: using ngrep, wireshark , tcpdump .. or maybe even Splunk Stream 🙂

Some examples here of using ngrep and tcpdump : https://stackoverflow.com/questions/9241391/how-to-capture-all-the-http-packets-using-tcpdump

Re: REST API Modular Input - 401 Client Unauthorized

jkat54 — Mon, 09 Oct 2017 23:17:35 GMT

I’ll second this approach

Re: REST API Modular Input - 401 Client Unauthorized

hpintelliflo — Tue, 10 Oct 2017 15:37:16 GMT

Guys - i think i have worked it out ... Nest does a 307 redirect. I suspect the module is not sending the headers on the redirect request. This Python is working

import httplib

headers = {"Authorization": "Bearer c.{INSERT KEY}"}

conn = httplib.HTTPSConnection("developer-api.nest.com")
conn.request("GET", "/", "", headers)
response = conn.getresponse()

url2 = response.getheader("location")
url2 = url2[8:-1]

conn2 = httplib.HTTPSConnection(url2)
conn2.request("GET", "/", "", headers)
response2 = conn2.getresponse()

print response2.read()
conn.close()
conn2.close()

Any thoughts?

Re: REST API Modular Input - 401 Client Unauthorized

Damien_Dallimor — Tue, 10 Oct 2017 20:38:00 GMT

You need to add a custom response handler in rest_ta/bin/responsehandlers.py and declare this handler to be applied in your stanza setup.

This custom handler would process the response from request 1 , get the the url from the location header for request 2 and send off request 2.

Re: REST API Modular Input - 401 Client Unauthorized

hpintelliflo — Tue, 10 Oct 2017 20:58:08 GMT

Awesome. Many thanks indeed for all your help on this one!