All Apps and Add-ons

Why does only first request works with REST API OAuth2 authentication?

Dohrendorf_Cons
Path Finder

Hello Splunkers,

I am currently running into a problem with the REST Api and OAuth2 authentication. I am trying to connect to a Google Analytics Endpoint. Everything is going great until the second request is fired off, which then fails every time.

For clarification: I setup a REST Modular Input, with ClientId, Client Secret, Access Token and Refresh Token and save it. I then see the data I expect, coming in to the specified index. However the second request then fails. The access token is still valid at that time. If I open the config again and save it, without changing anything, I get another succesful request and after that failed ones again.

My config looks like this:

[rest://<Name>]
auth_type = oauth2
endpoint = https://www.googleapis.com/analytics/v3/management/accountSummaries
http_method = GET
index_error_response_codes = 1
oauth2_token_type = 
response_type = text
sequential_mode = 0
sourcetype = _json
streaming_request = 0
oauth2_refresh_props = grant_type=refresh_token
oauth2_refresh_url = https://accounts.google.com/o/oauth2/token
oauth2_access_token = [****]
oauth2_client_id = [****]
oauth2_client_secret = [****]
oauth2_refresh_token = [****]
index = googleanalytics
polling_interval = 15

After the first succesful request I get this error response:

http_error_code = 401 error_message = {"error":{"errors":[{"domain":"global","reason":"required","message":"Login Required","locationType":"header","location":"Authorization"}],"code":401,"message":"Login Required"}}

I already experimented with several refresh parameters, including but not limited to (I have been at this for so long I am beginning to forget what I already tried and what I didn't 😞 ) :

approval_prompt = [force|auto]
grant_type = [refresh_token|client|implicit|credentials]
access_type = [offline|online]

Any idea on what I need to do to get more than one succesful request would be massively appreciated.

Labels (2)

kulrajatwal
Explorer

I ended up using the authhandlers.py script in $SPLUNK_HOME/etc/apps/rest_ta/bin

Used python to get a new token every hour, referenced that function in the ui for rest.

0 Karma

d646800
Explorer

Hi there, has this been resolved? I am currently facing now..

0 Karma

kulrajatwal
Explorer

Hi I am having a simular issue.  My token expires after 1 hour but it was failing after the first attempt.

 

I found in $SPLUNK_HOME/etc/apps/rest_ta/bin/rest.py

 

token["expires_in"] = "5"

So it expires in 5 seconds, i set this to 3600 so it uses the same token for an hour and that works, only problem now is its not refreshing the token as timing out going to the refresh token URL.  The proxy and routig to the url is fine so I think its something in the python code in the rest addon. 

 

Any help appreciated.

0 Karma

chkrug
Loves-to-Learn

Having the same issue as well. The app works for about an hour or so then it stops ingesting. We use refresh tokens that expire.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

Yes, there is a problem with oauth2 in rest TA.

————————————
If this helps, give a like below.
0 Karma

Damien_Dallimor
Ultra Champion

In SPLUNK_HOME/etc/apps/rest_ta/bin/rest.py can you set the logging to debug.

Look for this line :

logging.root.setLevel(logging.ERROR)

Change to :

logging.root.setLevel(logging.DEBUG)

Then see if you get any more useful debugging log messages to help diagnose what's going on.

Log messages searchable with this SPL : index=_internal ExecProcessor rest.py

0 Karma

Damien_Dallimor
Ultra Champion

That is not ideal as that will circumvent the Requests (python library being used) OAuth 2 session handling logic.

Any logs as per my previous reply ?

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Hi Damien, I'm experiencing the same problem. The requests work once after I set it up and after that, I'm getting error messages. If i open the input settings and save it again with any change it works again once and then back to the error messages I tried to raise the log level like you suggested but I see no difference in the logs.

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

diogofgm
SplunkTrust
SplunkTrust

After the sucessfull event i then get this:

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request: 

and after that all requests are returning errors

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

Dohrendorf_Cons
Path Finder

Any news on this?

0 Karma

thambisetty
SplunkTrust
SplunkTrust

It could be the proxy issue.

————————————
If this helps, give a like below.
0 Karma

Dohrendorf_Cons
Path Finder

Sorry took a while but here it is:

07-22-2016 08:46:29.392 +0200 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request: 

host = XXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/22/16

8:46:29.108 AM

07-22-2016 08:46:29.108 +0200 INFO  ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): accounts.google.com

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/22/16

`
8:46:03.946 AM

07-22-2016 08:46:03.946 +0200 INFO  ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): www.googleapis.com

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/22/16

8:45:58.783 AM

07-22-2016 08:45:58.783 +0200 INFO  ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:50:44.688 AM

07-21-2016 09:50:44.688 +0200 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request: 

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:50:44.151 AM

07-21-2016 09:50:44.151 +0200 INFO  ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): accounts.google.com

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:50:28.850 AM

07-21-2016 09:50:28.850 +0200 INFO  ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): www.googleapis.com

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:50:28.686 AM

07-21-2016 09:50:28.686 +0200 INFO  ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:17:20.138 AM

07-21-2016 09:17:20.138 +0200 INFO  ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:17:20.138 AM

07-21-2016 09:17:20.138 +0200 INFO  ExecProcessor - Removing status item "/opt/splunk/etc/apps/rest_ta/bin/rest.py (rest://XXXXXXXXXX) (isModInput=yes)

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:17:20.137 AM

07-21-2016 09:17:20.137 +0200 INFO  ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py

host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd

7/21/16

9:17:00.400 AM

07-21-2016 09:17:00.400 +0200 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" HTTP Request error: 404 Client Error: Not Found

They don't seem too useful to me, but maybe they tell you something.

Best regards and thanks for looking into this.

0 Karma

Dohrendorf_Cons
Path Finder

Ok,

I found out, that you can get it to work if you add the Access Token to the Custom Header field as

access_token=XXXXXX

Of course, doing it that way, means it stops working as soon as the current access_token has become invalid, since the refresh process is not working.

I would have thought that this is added to the request I am sending automatically and feel completely lost as to how I could get this to update automatically?

0 Karma

woodcock
Esteemed Legend

I would open a support case.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The REST Modular Input app is not splunk supported.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...