Why is Microsoft Office 365 O365 Add-On Data collection is having connection Failure?

[tnhawkman]

We have loaded the latest Office 365 Add-on. The configuration has been completed. However no data is coming in.
After changing the logging to Debug, I was able to see some info BELOW. It appears may be a permissions issue, but we have double checked everything there.

I have a case open, but any help would be greatly appreciated.

6/21/18 
3:13:44.557 PM  
2018-06-21 15:13:44,557 level=INFO pid=26767 tid=MainThread logger=splunksdc.collector pos=collector.py:run:248 | | message="Modular input exited." 
host =  REMOVED source =    /opt/splunk/var/log/splunk/splunk_ta_o365_management_activity_AzureAD.log sourcetype =  splunk:ta:o365:log 
6/21/18 
3:13:44.551 PM  
2018-06-21 15:13:44,551 level=ERROR pid=26767 tid=MainThread logger=splunk_ta_o365.modinputs.management_activity pos=utils.py:wrapper:67 | start_time=1529608423 datainput="AzureAD" | message="Data input was interrupted by an unhandled exception." 
Traceback (most recent call last): 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunksdc/utils.py", line 65, in wrapper 
return func(*args, **kwargs) 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/management_activity.py", line 91, in run 
executor.run(adapter) 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunksdc/batch.py", line 47, in run 
for jobs in delegate.discover(): 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/management_activity.py", line 114, in discover 
if not subscription.is_enabled(session): 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/common/portal.py", line 140, in is_enabled 
response = self._perform(session, 'GET', '/subscriptions/list') 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/common/portal.py", line 158, in _perform 
return self._request(session, method, url, kwargs) 
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/common/portal.py", line 170, in _request 
raise O365PortalError(response) 
O365PortalError: 401:{"error":{"code":"AF10001","message":"The permission set () sent in the request does not include the expected permission."}} 
Collapse 
host = REMOVED  source =    /opt/splunk/var/log/splunk/splunk_ta_o365_management_activity_AzureAD.log sourcetype =  splunk:ta:o365:log 
6/21/18 
3:13:44.425 PM  
2018-06-21 15:13:44,425 level=DEBUG pid=26767 tid=MainThread logger=splunk_ta_o365.common.portal pos=portal.py:_request:166 | start_time=1529608423 datainput="AzureAD" | message="Calling management activity API." url="https://manage.office365.us/api/v1.0/REMOVED/activity/feed/subscriptions/list" params={'PublisherIdentifier': u'REMOVED'} 
host =  REMOVED source =    /opt/splunk/var/log/splunk/splunk_ta_o365_management_activity_AzureAD.log sourcetype =  splunk:ta:o365:log 
6/21/18 
3:13:44.424 PM  
2018-06-21 15:13:44,424 level=INFO pid=26767 tid=MainThread logger=splunk_ta_o365.common.portal pos=portal.py:get_token_by_psk:92 | start_time=1529608423 datainput="AzureAD" | message="Acquire access token success." expires_on=1529612024

[jaxjohnny2000]

we get exactly the same, where did they "grant permissions"

2018-12-18 18:12:20,645 level=ERROR pid=77680 tid=MainThread logger=splunk_ta_o365.modinputs.management_activity pos=utils.py:wrapper:67 | start_time=1545156724 datainput="management_activity_audit_azure_ad" | message="Data input was interrupted by an unhandled exception."
Traceback (most recent call last):
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunksdc/utils.py", line 65, in wrapper
return func(*args, **kwargs)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/management_activity.py", line 91, in run
executor.run(adapter)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunksdc/batch.py", line 62, in run
delegate.done(job, result)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/management_activity.py", line 148, in done
self._ingest_content_blob(content, result)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/management_activity.py", line 167, in _ingest_content_blob
self._event_writer.write_fileobj(data, source=content.uri)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunksdc/event_writer.py", line 160, in write_fileobj
self._write(data)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunksdc/event_writer.py", line 132, in _write
self._dev.write(data)
IOError: [Errno 32] Broken pipe

[tnhawkman · Accepted solution]

Follow up. We did find that the Office 365 admin did not press the "Grant Permissions" button within the Office 365 setup. This step is easily overlooked, but is required to function. Hope this helps someone else.

David

[richgalloway]

@tnhawkman, If your problem is resolved, please accept the answer.

[salles]

Thanks so much, this saved me a bunch of time!