Splunk Search

Microsoft Office 365 Reporting Add-on not obeying interval configuration

Communicator

Hi,

 

we are using version 1.2.4 on Splunk 7.3.7, and we noticed our interval setting of (interval=600 / 10 mins) is not being obeyed. We can see when it does make a successful connection and pull the logs, we see at the start of the connection "HTTP connection pooling" in the logs. However, what we see subsequently are continuous connections every minute or so. We tried a splunk restart to see if this made a difference but it hasn't changed it's behaviour. So now we see connections from every 30-40 mins or longer. 

Below is an example of the logs:

 

2020-11-23 15:42:12,824 INFO pid=32193 tid=MainThread file=splunk_rest_client.py:_request_handler:105 | Use HTTP connection pooling
2020-11-23 15:42:12,824 DEBUG pid=32193 tid=MainThread file=binding.py:get:677 | GET request to https://127.0.0.1:9001/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_... (body: {})
2020-11-23 15:42:12,826 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_new_conn:959 | Starting new HTTPS connection (1): 127.0.0.1:9001
2020-11-23 15:42:12,834 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:9001 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 5509
2020-11-23 15:42:12,835 DEBUG pid=32193 tid=MainThread file=binding.py:new_f:73 | Operation took 0:00:00.010786
2020-11-23 15:42:12,836 DEBUG pid=32193 tid=MainThread file=binding.py:get:677 | GET request to https://127.0.0.1:9001/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/ (body: {'count': -1, 'search': 'TA_MS_O365_Reporting_checkpointer', 'offset': 0})
2020-11-23 15:42:12,842 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:9001 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/?count=-1&search=TA_MS_O365_Reporting_checkpointer&offset=0 HTTP/1.1" 200 7403
2020-11-23 15:42:12,844 DEBUG pid=32193 tid=MainThread file=binding.py:new_f:73 | Operation took 0:00:00.007860
2020-11-23 15:42:12,852 DEBUG pid=32193 tid=MainThread file=binding.py:get:677 | GET request to https://127.0.0.1:9001/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Re... (body: {})
2020-11-23 15:42:12,857 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:9001 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/index_continuously_obj_checkpoint HTTP/1.1" 200 128
2020-11-23 15:42:12,857 DEBUG pid=32193 tid=MainThread file=binding.py:new_f:73 | Operation took 0:00:00.005903
2020-11-23 15:42:12,858 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | _Splunk_ Start date: 2020-11-23 14:21:59.057785, End date: 2020-11-23 14:31:59.057785
2020-11-23 15:42:12,858 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?\$filter=StartDate eq datetime'2020-11-23T14:21:59.057785Z' and EndDate eq datetime'2020-11-23T14:31:59.057785Z'
2020-11-23 15:42:12,863 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_new_conn:959 | Starting new HTTPS connection (1): reports.office365.com:443
2020-11-23 15:42:16,000 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?%5C$filter=StartDate%20eq%20datetime'2020-11-23T14:21:59.057785Z'%20and%20EndDate%20eq%20datetime'2020-11-23T14:31:59.057785Z' HTTP/1.1" 200 None
2020-11-23 15:42:16,073 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | _Splunk_ max date before getting message: 2020-11-23 14:21:59.057785
2020-11-23 15:42:16,893 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | _Splunk_ max date after getting messages: 2020-11-23 15:41:50.582546
2020-11-23 15:42:16,894 DEBUG pid=32193 tid=MainThread file=binding.py:post:750 | POST request to https://127.0.0.1:9001/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Re... (body: {'body': '[{"state": "{\\"max_date\\": \\"2020-11-23 15:41:50.582546\\"}", "_key": "index_continuously_obj_checkpoint"}]'})
2020-11-23 15:42:16,926 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:9001 "POST /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save HTTP/1.1" 200 39
2020-11-23 15:42:16,928 DEBUG pid=32193 tid=MainThread file=binding.py:new_f:73 | Operation took 0:00:00.033994
2020-11-23 15:42:16,928 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | _Splunk_ nextLink URL (@odata.nextLink): https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$skiptoken=1999
2020-11-23 15:42:16,928 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$skiptoken=1999
2020-11-23 15:42:16,932 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_new_conn:959 | Starting new HTTPS connection (1): reports.office365.com:443
2020-11-23 15:42:18,938 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$skiptoken=1999 HTTP/1.1" 200 None
2020-11-23 15:42:19,777 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | _Splunk_ max date after getting messages: 2020-11-23 15:41:50.582546
2020-11-23 15:42:19,778 DEBUG pid=32193 tid=MainThread file=binding.py:post:750 | POST request to https://127.0.0.1:9001/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Re... (body: {'body': '[{"state": "{\\"max_date\\": \\"2020-11-23 15:41:50.582546\\"}", "_key": "index_continuously_obj_checkpoint"}]'})
2020-11-23 15:42:19,827 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://127.0.0.1:9001 "POST /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/batch_save HTTP/1.1" 200 39
2020-11-23 15:42:19,829 DEBUG pid=32193 tid=MainThread file=binding.py:new_f:73 | Operation took 0:00:00.051431
2020-11-23 15:42:19,830 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | _Splunk_ nextLink URL (@odata.nextLink): https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$skiptoken=3999
2020-11-23 15:42:19,830 DEBUG pid=32193 tid=MainThread file=base_modinput.py:log_debug:288 | Endpoint URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$skiptoken=3999
2020-11-23 15:42:19,834 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_new_conn:959 | Starting new HTTPS connection (1): reports.office365.com:443
2020-11-23 15:42:21,872 DEBUG pid=32193 tid=MainThread file=connectionpool.py:_make_request:437 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$skiptoken=3999

 

thanks

Labels (1)
0 Karma