No data sent to Splunk while debug logs show successful connections to Microsoft graph API, and 'None' content length

[shannan2]

In an attempt to bring in some additional Azure AD data we have begun using the Microsoft Azure Add-on for Splunk, however we are not seeing any results actually come back to Splunk, and not seeing any errors in collection.

When enabling debug logging I can see that we are getting a http status code of 200, but a content length of 'None'

2020-03-20 19:44:33,920 DEBUG pid=62692 tid=MainThread file=connectionpool.py:_make_request:400 | https://graph.microsoft.com:443 "GET /beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+2020-03-13T19%3a29%3a45.102703Z+and+activityDateTime+le+2020-03-20T19%3a22%3a45.501341Z&$skiptoken=f207127ca72cc8e1dca1f7873280c23e_326040 HTTP/1.1" 200 None

And the python which generates the logs to see that the length is the final attribute of that log message in connectionpool.py:_make_request:400

log.debug("%s://%s:%s \"%s %s %s\" %s %s", self.scheme, self.host, self.port,
                  method, url, http_version, httplib_response.status,
                  httplib_response.length)

The service continues to run and get 200 responses, and even finding the next link to go to from @odata.nextLink within the JSON returned, even though the debug logging shows no content length.

2020-03-20 19:44:34,136 DEBUG pid=62692 tid=MainThread file=base_modinput.py:log_debug:286 | _Splunk_ nextLink URL (@odata.nextLink): https://graph.microsoft.com/beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+2020-03-13T19%3a29%3a45.102703Z+and+activityDateTime+le+2020-03-20T19%3a22%3a45.501341Z&$skiptoken=a05e0f88b9ed79f47b8955baefdf862c_327085

I have been able to replicate the action being used by the TA within Postman using the Microsoft Graph Collection and environment (https://docs.microsoft.com/en-us/graph/use-postman) where I can see plenty of data returned from each one of the URLs provided and I am using the exact same Azure AD application with the same client id and secret.

I am trying to dig through the actual python but I have not been able to find anything in relation to causing this issue yet.

[mbrose66]

So you’ve installed the newest version of the Azure Add-on (2.0.2) on both your ES Search Head (if you have one), Ad-hocs, and on a Heavy Forwarder, correct?

On your Heavy Forwarder, you aren’t running Version 8 of Splunk, correct? (Ideally 7.3.3). On the Heavy Forwarder in the Azure Add-On, you created two new inputs for the Azure AD Sign-Ins and the Azure AD Directory Audit, right? You can also make a third one for the Monitor Metrics if you want to bring those in as well. For each of those inputs the client ID and secret must be entered on the configuration tabs. I assume you've entered those and saved?

What is your interval setting at? Would recommend something like 300 or 180 seconds (usually 300).

I would also edit both input_module_MS_AAD_audit.py (audit.py) and input_module_MS_AAD_signins.py (signins.py) to correct the query times (change value to 5 minutes).

query_date = get_start_date(helper, check_point_key)
query_date_end = (datetime.datetime.utcnow() - datetime.timedelta(minutes=5)).strftime(‘%Y-%m-%dT%H:%M:%S.%fZ’)

There shouldn't be a need to restart Splunk after the change, but you can if you'd like.

Looking at the log you posted- your search time was very extensive. Searching between March 13 and March 20!

“GET /beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+2020-03-13T19%3a29%3a45.102703Z+and+activityDateTime+le+2020-03-20T19%3a22%3a45.501341Z&$skiptoken=f207127ca72cc8e1dca1f7873280c23e_326040 HTTP/1.1”

You would expect to see something more like this:

"GET /beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+2020-04-01T02%3a05%3a16.3181023Z+and+activityDateTime+le+2020-04-01T02%3a15%3a16.703416Z&$skiptoken=bc772c3a4143ceed5b8eb9acb6288b56_1045 HTTP/1.1" 200