How to get MESSAGE TRACE LOGS from Azure to Splunk

[jacknguyen]

Hi all, I have a big problem with my customer.

I try to get message trace logs from Azure for O365, following the Splunk doc, my account have all 3 roles:

Exchange Administrator 

Global Administrator 

Global Reader role

In my lab Splunk I try to trouble shoot by _internal and I have this. Any one know why and how to fix this?

this is sample log:

2023-07-08 20:00:18,077 level=ERROR pid=10564 tid=MainThread logger=splunk_ta_o365.modinputs.message_trace pos=__init__.py:run:376 | datainput=b'messagetrace' start_time=1688821215 | message="An error occurred while collecting data" stack_info=True
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/message_trace/__init__.py", line 371, in run
    self._collect_events(app)
  File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/message_trace/__init__.py", line 145, in _collect_events
    self._get_events_continuous(app)
  File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/message_trace/__init__.py", line 216, in _get_events_continuous
    self._process_messages(start_date, end_date)
  File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/message_trace/__init__.py", line 283, in _process_messages
    message_response = self._get_messages(microsoft_trace_url)
  File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/message_trace/__init__.py", line 270, in _get_messages
    raise e
  File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/message_trace/__init__.py", line 262, in _get_messages
    response.raise_for_status()
  File "/opt/splunk/etc/apps/splunk_ta_o365/lib/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error:  for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2023-07-03T12:54:27Z'%20and%20EndDate%20eq%20datetime'2023-07-03T13:54:27Z'