Hello Splunkers,
I am currently running into a problem with the REST Api and OAuth2 authentication. I am trying to connect to a Google Analytics Endpoint. Everything is going great until the second request is fired off, which then fails every time.
For clarification: I setup a REST Modular Input, with ClientId, Client Secret, Access Token and Refresh Token and save it. I then see the data I expect, coming in to the specified index. However the second request then fails. The access token is still valid at that time. If I open the config again and save it, without changing anything, I get another succesful request and after that failed ones again.
My config looks like this:
[rest://<Name>]
auth_type = oauth2
endpoint = https://www.googleapis.com/analytics/v3/management/accountSummaries
http_method = GET
index_error_response_codes = 1
oauth2_token_type =
response_type = text
sequential_mode = 0
sourcetype = _json
streaming_request = 0
oauth2_refresh_props = grant_type=refresh_token
oauth2_refresh_url = https://accounts.google.com/o/oauth2/token
oauth2_access_token = [****]
oauth2_client_id = [****]
oauth2_client_secret = [****]
oauth2_refresh_token = [****]
index = googleanalytics
polling_interval = 15
After the first succesful request I get this error response:
http_error_code = 401 error_message = {"error":{"errors":[{"domain":"global","reason":"required","message":"Login Required","locationType":"header","location":"Authorization"}],"code":401,"message":"Login Required"}}
I already experimented with several refresh parameters, including but not limited to (I have been at this for so long I am beginning to forget what I already tried and what I didn't 😞 ) :
approval_prompt = [force|auto]
grant_type = [refresh_token|client|implicit|credentials]
access_type = [offline|online]
Any idea on what I need to do to get more than one succesful request would be massively appreciated.
I ended up using the authhandlers.py script in $SPLUNK_HOME/etc/apps/rest_ta/bin
Used python to get a new token every hour, referenced that function in the ui for rest.
Hi there, has this been resolved? I am currently facing now..
Hi I am having a simular issue. My token expires after 1 hour but it was failing after the first attempt.
I found in $SPLUNK_HOME/etc/apps/rest_ta/bin/rest.py
token["expires_in"] = "5"
So it expires in 5 seconds, i set this to 3600 so it uses the same token for an hour and that works, only problem now is its not refreshing the token as timing out going to the refresh token URL. The proxy and routig to the url is fine so I think its something in the python code in the rest addon.
Any help appreciated.
Having the same issue as well. The app works for about an hour or so then it stops ingesting. We use refresh tokens that expire.
Yes, there is a problem with oauth2 in rest TA.
In SPLUNK_HOME/etc/apps/rest_ta/bin/rest.py
can you set the logging to debug.
Look for this line :
logging.root.setLevel(logging.ERROR)
Change to :
logging.root.setLevel(logging.DEBUG)
Then see if you get any more useful debugging log messages to help diagnose what's going on.
Log messages searchable with this SPL : index=_internal ExecProcessor rest.py
That is not ideal as that will circumvent the Requests (python library being used) OAuth 2 session handling logic.
Any logs as per my previous reply ?
Hi Damien, I'm experiencing the same problem. The requests work once after I set it up and after that, I'm getting error messages. If i open the input settings and save it again with any change it works again once and then back to the error messages I tried to raise the log level like you suggested but I see no difference in the logs.
After the sucessfull event i then get this:
ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:
and after that all requests are returning errors
Any news on this?
It could be the proxy issue.
Sorry took a while but here it is:
07-22-2016 08:46:29.392 +0200 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:
host = XXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/22/16
8:46:29.108 AM
07-22-2016 08:46:29.108 +0200 INFO ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): accounts.google.com
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/22/16
`
8:46:03.946 AM
07-22-2016 08:46:03.946 +0200 INFO ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): www.googleapis.com
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/22/16
8:45:58.783 AM
07-22-2016 08:45:58.783 +0200 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:50:44.688 AM
07-21-2016 09:50:44.688 +0200 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:50:44.151 AM
07-21-2016 09:50:44.151 +0200 INFO ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): accounts.google.com
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:50:28.850 AM
07-21-2016 09:50:28.850 +0200 INFO ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Starting new HTTPS connection (1): www.googleapis.com
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:50:28.686 AM
07-21-2016 09:50:28.686 +0200 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:17:20.138 AM
07-21-2016 09:17:20.138 +0200 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:17:20.138 AM
07-21-2016 09:17:20.138 +0200 INFO ExecProcessor - Removing status item "/opt/splunk/etc/apps/rest_ta/bin/rest.py (rest://XXXXXXXXXX) (isModInput=yes)
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:17:20.137 AM
07-21-2016 09:17:20.137 +0200 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py
host = XXXXX
source = /opt/splunk/var/log/splunk/splunkd.log
sourcetype = splunkd
7/21/16
9:17:00.400 AM
07-21-2016 09:17:00.400 +0200 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" HTTP Request error: 404 Client Error: Not Found
They don't seem too useful to me, but maybe they tell you something.
Best regards and thanks for looking into this.
Ok,
I found out, that you can get it to work if you add the Access Token to the Custom Header field as
access_token=XXXXXX
Of course, doing it that way, means it stops working as soon as the current access_token has become invalid, since the refresh process is not working.
I would have thought that this is added to the request I am sending automatically and feel completely lost as to how I could get this to update automatically?
I would open a support case.
The REST Modular Input app is not splunk supported.