topic Re: microsoft office 365 reporting add-on stopped getting data in All Apps and Add-ons

microsoft office 365 reporting add-on stopped getting data

martinnepolean — Thu, 09 Jul 2020 15:43:12 GMT

I am using version 1.1.0, the Log file doesnt show any error but I could it tries to get every 2 seconds, where the polling is 60 seconds.

2020-07-09 10:37:15,846 DEBUG pid=14036 tid=MainThread file=connectionpool.py:_new_conn:809 | Starting new HTTPS connection (1): reports.office365.com
2020-07-09 10:37:17,849 DEBUG pid=14036 tid=MainThread file=connectionpool.py:_make_request:400 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-09T07%3A22%3A05.982209Z'%20and%20EndDate%20eq%20datetime'2020-07-09T07%3A52%3A05.982209Z'&$skiptoken=3999 HTTP/1.1" 200 51899
2020-07-09 10:37:17,882 DEBUG pid=14036 tid=MainThread file=base_modinput.py:log_debug:286 | Next URL is https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-09T07%3A22%3A05.982209Z'%20and%20EndDate%20eq%20datetime'2020-07-09T07%3A52%3A05.982209Z'&$skiptoken=5999
2020-07-09 10:37:17,883 DEBUG pid=14036 tid=MainThread file=base_modinput.py:log_debug:286 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-09T07%3A22%3A05.982209Z'%20and%20EndDate%20eq%20datetime'2020-07-09T07%3A52%3A05.982209Z'&$skiptoken=5999
2020-07-09 10:37:17,883 INFO pid=14036 tid=MainThread file=setup_util.py:log_info:114 | Proxy is not enabled!
2020-07-09 10:37:17,884 DEBUG pid=14036 tid=MainThread file=connectionpool.py:_new_conn:809 | Starting new HTTPS connection (1): reports.office365.com
2020-07-09 10:37:19,820 DEBUG pid=14036 tid=MainThread file=connectionpool.py:_make_request:400 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-09T07%3A22%3A05.982209Z'%20and%20EndDate%20eq%20datetime'2020-07-09T07%3A52%3A05.982209Z'&$skiptoken=5999 HTTP/1.1" 200 50410
2020-07-09 10:37:19,852 DEBUG pid=14036 tid=MainThread file=base_modinput.py:log_debug:286 | Next URL is https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-09T07%3A22%3A05.982209Z'%20and%20EndDate%20eq%20datetime'2020-07-09T07%3A52%3A05.982209Z'&$skiptoken=7999
2020-07-09 10:37:19,853 DEBUG pid=14036 tid=MainThread file=base_modinput.py:log_debug:286 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-09T07%3A22%3A05.982209Z'%20and%20EndDate%20eq%20datetime'2020-07-09T07%3A52%3A05.982209Z'&$skiptoken=7999

Re: microsoft office 365 reporting add-on stopped getting data

martinnepolean — Fri, 10 Jul 2020 06:39:49 GMT

Upgraded to the latest version, and created a new profile, still, the issue exists. Any help?

Re: microsoft office 365 reporting add-on stopped getting data

lim2 — Fri, 10 Jul 2020 22:45:26 GMT

Also having this issue on Splunk 8.0.3 with version 1.2.1 July10 (https://splunkbase.splunk.com/app/3720/#/overview). Tried with version 1.2.0 also same issue. Came from version 1.1.0 which was working fine until 6/20/2020
2020-07-10 18:39:27,479 DEBUG pid=25523 tid=MainThread file=base_modinput.py:log_debug:288 | Start date: 2020-06-20 09:04:53, End date: 2020-06-20 09:14:53
2020-07-10 18:39:27,479 DEBUG pid=25523 tid=MainThread file=base_modinput.py:log_debug:288 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2020-06-20T09:04:53Z' and EndDate eq datetime'2020-06-20T09:14:53Z'
2020-07-10 18:39:27,479 INFO pid=25523 tid=MainThread file=setup_util.py:log_info:117 | Proxy is not enabled!
2020-07-10 18:39:27,480 DEBUG pid=25523 tid=MainThread file=connectionpool.py:_new_conn:959 | Starting new HTTPS connection (1): reports.office365.com:443
2020-07-10 18:39:27,758 DEBUG pid=25523 tid=MainThread file=connectionpool.py:_make_request:437 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-06-20T09:04:53Z'%20and%20EndDate%20eq%20datetime'2020-06-20T09:14:53Z' HTTP/1.1" 400 102
2020-07-10 18:39:27,760 ERROR pid=25523 tid=MainThread file=base_modinput.py:log_error:309 | HTTP Request error: 400 Client Error: Bad Request for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-06-20T09:04:53Z'%20and%20EndDate%20eq%20datetime'2020-06-20T09:14:53Z

I could use the browser and get to https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2020-06-20T09:04:53Z' and EndDate eq datetime'2020-06-20T09:14:53Z'
and download the MessageTrace json file ok

@SplunkWorks development team, help please.

Thanks.

Re: microsoft office 365 reporting add-on stopped getting data

lim2 — Sun, 12 Jul 2020 23:24:44 GMT

Issue is resolved for my case. Was trying to get O365 MessageTrace logs which are older than 10 days (https://docs.microsoft.com/en-us/powershell/module/exchange/get-messagetrace?view=exchange-ps) so was getting the http 400 bad request code .

Did a completely new install of the newest TA version 1.2.1 and set the Start date/time to 10 days ago 2020-07-02T19:00:00 . Lesson learnt to have an alert for no O365 MessageTrace log because cannot get logs older than 10 days.
Log of the successful run:

2020-07-12 18:53:20,548 DEBUG pid=9848 tid=MainThread file=base_modinput.py:log_debug:288 | Start date: 2020-07-02 19:00:00, End date: 2020-07-02 20:00:00
2020-07-12 18:53:20,548 DEBUG pid=9848 tid=MainThread file=base_modinput.py:log_debug:288 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2020-07-02T19:00:00Z' and EndDate eq datetime'2020-07-02T20:00:00Z'
2020-07-12 18:53:20,548 INFO pid=9848 tid=MainThread file=setup_util.py:log_info:117 | Proxy is not enabled!
2020-07-12 18:53:20,549 DEBUG pid=9848 tid=MainThread file=connectionpool.py:_new_conn:959 | Starting new HTTPS connection (1): reports.office365.com:443
2020-07-12 18:53:23,059 DEBUG pid=9848 tid=MainThread file=connectionpool.py:_make_request:437 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-02T19:00:00Z'%20and%20EndDate%20eq%20datetime'2020-07-02T20:00:00Z' HTTP/1.1" 200 None
2020-07-12 18:53:23,171 DEBUG pid=9848 tid=MainThread file=base_modinput.py:log_debug:288 | Next URL is https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-07-02T19%3A00%3A00Z'%20and%20EndDate%20eq%20datetime'2020-07-02T20%3A00%3A00Z'&$skiptoken=1999

....
2020-07-12 18:54:11,931 DEBUG pid=9848 tid=MainThread file=connectionpool.py:_make_request:437 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$
filter=StartDate%20eq%20datetime'2020-07-02T19%3A00%3A00Z'%20and%20EndDate%20eq%20datetime'2020-07-02T20%3A00%3A00Z'&$skiptoken=51999 HTTP/1.1" 200 37202
2020-07-12 18:54:11,939 DEBUG pid=9848 tid=MainThread file=base_modinput.py:log_debug:288 | Number of messages returned: 52751
2020-07-12 18:54:11,939 DEBUG pid=9848 tid=MainThread file=base_modinput.py:log_debug:288 | Max date before getting message: 2020-07-02 19:00:00
2020-07-12 18:54:20,879 DEBUG pid=9848 tid=MainThread file=base_modinput.py:log_debug:288 | Max date after getting messages: 2020-07-02 19:59:59.850906
2020-07-12 18:54:20,879 DEBUG pid=9848 tid=MainThread file=binding.py:post:750 | POST request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save (body: {'body': '[{"_key": "Company_O365_Reporting_checkpoint", "state": "{\\"max_date\\": \\"2020-07-02 19:59:59.850906\\"}"}]'})
2020-07-12 18:54:20,884 DEBUG pid=9848 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:8089 "POST /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save HTTP/1.1" 200 45
2020-07-12 18:54:20,885 DEBUG pid=9848 tid=MainThread file=binding.py:new_f:73 | Operation took 0:00:00.005899

Re: microsoft office 365 reporting add-on stopped getting data

martinnepolean — Mon, 13 Jul 2020 11:12:17 GMT

@lim2 Yes, even I get 400 bad request error when I give data which is 7 days older. But here, the data collection is stopped on 9th July and I am trying to get from 9th data and I am not seeing 400 bad requests in the log.

Re: microsoft office 365 reporting add-on stopped getting data

lim2 — Mon, 13 Jul 2020 12:12:30 GMT

@martinnepolean
I've to constanty check on the latest 1.2.1TA (having to disable & enable the input) & running
tail -500f /opt/splunk/var/log/splunk/ta_ms_o365_reporting_ms_o365_message_trace.log|egrep "Start date|return|batch_save|max_date"
Are you seeing any "Number of messages returned:" and "batch_save" ?
2020-07-13 08:05:38,881 DEBUG pid=28145 tid=MainThread file=base_modinput.py:log_debug:288 | Start date: 2020-07-08 19:52:32.511000, End date: 2020-07-08 20:52:32.511000
2020-07-13 08:06:45,692 DEBUG pid=28145 tid=MainThread file=base_modinput.py:log_debug:288 | Number of messages returned: 63039
2020-07-13 08:07:00,460 DEBUG pid=28145 tid=MainThread file=binding.py:post:750 | POST request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save (body: {'body': '[{"_key": "Company_MO365_checkpoint", "state": "{\\"max_date\\": \\"2020-07-08 20:52:32.465485\\"}"}]'})
2020-07-13 08:07:00,489 DEBUG pid=28145 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:8089 "POST /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save HTTP/1.1" 200 45
2020-07-13 08:07:08,528 DEBUG pid=28666 tid=MainThread file=base_modinput.py:log_debug:288 | Start date: 2020-07-08 20:52:32.465485, End date: 2020-07-08 21:52:32.465485

Re: microsoft office 365 reporting add-on stopped getting data

martinnepolean — Mon, 13 Jul 2020 12:21:32 GMT

No, there is no - "Number of messages returned" in my log.

Re: microsoft office 365 reporting add-on stopped getting data

lim2 — Mon, 13 Jul 2020 13:00:10 GMT

@martinnepolean

May need to reset the TA_MS_O365_Reporting_checkpointer 's max_date:

To check:
curl -k -u admin -H "Content-Type: application/json" -X GET https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/

To set:
curl -k -u admin -H "Content-Type: application/json" -X POST https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save -d '[{ "state": "{\"max_date\": \"2020-07-09 07:22:00\"}", "_user": "nobody", "_key": "your_inputname_checkpoint" } ]

Re: microsoft office 365 reporting add-on stopped getting data

martinnepolean — Tue, 14 Jul 2020 08:03:58 GMT

Issue resolved after recreating new profile and set the start date to today. Seems there was some issue in getting the 9th July data. But there is no log about it.

Re: microsoft office 365 reporting add-on stopped getting data

JoeT123 — Mon, 05 Oct 2020 17:51:00 GMT

Hoping someone can help as we are experiencing the same issue.

502 ERROR pid=1468 tid=MainThread file=base_modinput.py:log_error:309 | HTTP Request error: 401 Client Error: Unauthorized for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2020-10-01T00:00:00Z'%20and%20EndDate%20eq%20datetime'2020-10-01T01:00:00Z'

We are running TA-MS_O365_Reporting version 1.2.1 from our cloud add-hoc search head and are receiving the above error. We tried this from our on-premise HF but also received same results. Wondering if this is a permissions issue into 0365. Is there any specific permissions for the service account for this to work?

Re: microsoft office 365 reporting add-on stopped getting data

ASierra — Thu, 01 Jul 2021 14:53:25 GMT

For anyone getting errors, you should update the query from

OLD URL:

https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate

NEW URL:

https://outlook.office365.com/ecp/MessageTrace/MessageTraceResults.aspx?$filter=StartDate

The url base in the SPL should also be udpated to: https://outlook.office365.com/ecp

Re: microsoft office 365 reporting add-on stopped getting data

TheCityRich2 — Fri, 12 Nov 2021 20:20:46 GMT

Has anyone had any issues with this add-on since upgrading the Splunk upgrade to python3? I was troubleshooting this today with the guidance of this thread but every time I change the config, the actual http GET request to office365 does not change. I removed the add-on then tried to re-install but the verification fails now.

I'm just wondering if anyone else is having similar issues.