Hello Splunkers,
currently I am struggling with the REST-TA Modular Input. We are trying to access the WLAN Controller of Ubiquity Unifi WLAN Access Points, which offers a REST API. We want to retrieve the event log.
The sequence of actions needed to retrieve the event log is:
1. login into the Unifi Controller with https://unifi.domain.de:7777/login?username=XXX&login=login&password=YYY
2. retrieve the event log with https://unifi.domain.de:7777/api/stat/event
, this returns a JSON structure
3. logout with https://unifi.domain.de:7777/logout
The difference to all other questions regarding REST-TA is: Before actually GETting the information requested, a https-Login-packet is required, and afterwards a https-Logout-packet must be sent to the REST-API-provider.
This is the inputs.conf stanza:
[rest://Unifi WLAN Events]
auth_type = custom
custom_auth_handler = MyUnifyAuth
endpoint = https://unifi.domain.de:7777/api/stat/event
http_method = GET
index = test
index_error_response_codes = 1
polling_interval = 120
response_handler = MyUnifyResponse
response_type = json
sourcetype = _json
streaming_request = 0
disabled = 1
And this is the custom authhandlers.py (only additional lines to the shipped file, code is taken mostly from Unifi-API examples at github):
import urllib2
import cookielib
#login for Unifi WLAN controller class MyUnifyAuth(AuthBase):
def __init__(self,**args):
cj = cookielib.CookieJar()
self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
pass
def __call__(self, r):
loginurl = 'https://unifi.domain.de:7777/login?username=XXX&login=login&password=YYY'
self.opener.open(loginurl).read()
return r
And the custom responsehandlers.py basically is the default handler, extended with the logout:
class MyUnifyResponse:
def __init__(self,**args):
pass
def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
self.opener.open('https://unifi.domain.de:7777/logout').read()
cookies = response_object.cookies
if cookies:
req_args["cookies"] = cookies
print_xml_stream(raw_response_output)
Now for the problems/questions:
ExecProcessor - message from "python /export/server/splunk/etc/apps/rest_ta/bin/rest.py" HTTP Request error: 401 Client Error: Unauthorized
Thanks in advance for helpful hints!!
Best regards,
Stephan
I'm making a very wild guess here as I know nothing about Unifi. So try the below ideas.
[rest://Unifi WLAN Events]
auth_type = custom
custom_auth_handler = MyUnifyAuth
custom_auth_handler_args = username=foo,password=goo,login_url=https://unifi.domain.de:7777/login
endpoint = https://unifi.domain.de:7777/api/stat/event
http_method = GET
index = test
index_error_response_codes = 1
polling_interval = 120
response_handler = MyUnifyResponse
response_handler_args = logout_url=https://unifi.domain.de:7777/logout
response_type = json
sourcetype = _json
streaming_request = 0
disabled = 1
class MyUnifyAuth(AuthBase):
def __init__(self,**args):
self.username = args['username']
self.password = args['password']
self.login_url = args['login_url']
pass
def __call__(self, r):
payload = {'username': self.username, 'login': 'login','password':self.password}
login_response = requests.get(self.login_url,params=payload,verify=false)
cookies = login_response.cookies
if cookies:
r.cookies = cookies
return r
class MyUnifyResponse:
def __init__(self,**args):
self.logout_url = args['logout_url']
pass
def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
cookies = response_object.cookies
if cookies:
req_args["cookies"] = cookies
requests.get(self.logout_url,verify=false,cookies=cookies)
print_xml_stream(raw_response_output)
Hi Damien,
thanks for the very quick answer!! Now I understand the data flow much better. I had to add verify=False
to the requests.get, otherwise SSL errors are thrown, because only self signed certificates are used. Otherwise I am using exactl your code.
I have found out that the login sets a session cookie (destroyed at the end of a session) named "unifises" with a hex string, probably a kind of session id.
I still get an 401 error, client unauthorized. Since the cookie is returned in r.cookies probably the session is not reused inside the rest-ta? Currently I can't imagine a reason for the 401.
If it is a certification validation problem: How do I propagate the verify=False to the get-Request of the endpoint-URL?
Thanks again,
Stephan
I only have this to go off : https://github.com/calmh/unifi-api/blob/master/unifi/controller.py
I see that there is some dynamic URL construction going on , is https://unifi.domain.de:7777/api/stat/event the correct URL ?
Can you trace the requests/responses being sent/received (ie: using a tool like wireshark) , to ensure that the HTTP GET looks correct ?
The URLs for login, logout and REST-API-call to get the events have been checked inside a Firefox-browser, the browser-window displays the json-list of events. These are the URLs as they actually are constructed by the controller.py from github, what we have used to understand the required flow of https-packets.
Wireshark is not so easy, everything is ssl-encoded...
Is the 401 occurring on the Login HTTP GET or the Endpoint HTTP GET ?
OK, I am pretty much shure, that this is related to the session-only duration of the cookie.
What I did inside the MyUnifiAuth class to test this:
- request.get with Login-String, this returned 200
- request.get with event-API-call, this returned 401, and a json-error "api.err.LoginRequired"
Seems that this are actually two sessions...
Thanks for supporting us!!
I think, we should consider other (outside of Splunk) methods to retrieve the event list using an intermediate file.