MS Azure Active Directory Reporting Add-on for Splunk: Why are there errors after upgrading to 1.1?

[wstarowicz]

Hi I have following errors after upgrading to 1.1 version. It's a bit strange as logs seems to be ingested.

2019-04-16 08:13:18,919 ERROR pid=2020 tid=MainThread file=splunk_rest_client.py:request:144 | Failed to issue http request=POST to url=https://127.0.0.1:8089/servicesNS/nobody/TA-MS-AAD/storage/collections/data/TA_MS_AAD_checkpointer/batch_save, error=Traceback (most recent call last):
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/splunk_rest_client.py", line 140, in request
    verify=verify, proxies=proxies, cert=cert, **kwargs)
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/requests/adapters.py", line 426, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))

    2019-04-16 08:13:18,922 ERROR pid=2020 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
    Traceback (most recent call last):
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 127, in stream_events
        self.collect_events(ew)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_signins.py", line 76, in collect_events
        input_module.collect_events(self, ew)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_signins.py", line 81, in collect_events
        helper.save_check_point(check_point_key, max_signinDateTime)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 529, in save_check_point
        self.ckpt.update(key, state)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/utils.py", line 154, in wrapper
        return func(*args, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/modular_input/checkpointer.py", line 208, in update
        self._collection_data.batch_save(record)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/client.py", line 3719, in batch_save
        return json.loads(self._post('batch_save', headers=KVStoreCollectionData.JSON_HEADER, body=data).body.read())
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/client.py", line 3610, in _post
        return self.service.post(self.path + url, owner=self.owner, app=self.app, sharing=self.sharing, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/binding.py", line 287, in wrapper
        return request_fun(self, *args, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/binding.py", line 69, in new_f
        val = f(*args, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/binding.py", line 738, in post
        response = self.http.post(path, all_headers, **query)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/binding.py", line 1201, in post
        return self.request(url, message)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/splunklib/binding.py", line 1218, in request
        response = self.handler(url, message, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/splunk_rest_client.py", line 140, in request
        verify=verify, proxies=proxies, cert=cert, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/requests/sessions.py", line 468, in request
        resp = self.send(prep, **send_kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/requests/sessions.py", line 576, in send
        r = adapter.send(request, **kwargs)
      File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/packages/requests/adapters.py", line 426, in send
        raise ConnectionError(err, request=request)
    ConnectionError: ('Connection aborted.', BadStatusLine("''",))

[wstarowicz]

Thanks, now it seems to be ok!

[jconger · Accepted solution]

This looks to be an error retrieving the checkpoint data. The checkpoint contains the timestamp of the last Azure AD event read to use as a starting point for the next query. Try the following search to see what data is in the checkpoint:

| inputlookup AAD_checkpoint_lookup | eval key=_key

Another thing to try is disabling your input(s) and creating a new one with the start date/time specified. This will create a new checkpoint entry.