All Apps and Add-ons

Unable to receive the logs from Microsoft Azure Active Directory Reporting Add-on for Splunk 1.1.0

subbarayudu
New Member

Team,

In one of my setup, I have previous version of App, previous version app is unable to collect the complete sigin-in logs that is visible in Azure Portal.

As I noticed new version of app was released, Installed the new version of App in another setup,configured the inputs(used same config key from working setup) noticed the below error logs,

Kindly help to fix the issue

2019-04-25 10:27:56,177 DEBUG pid=6441 tid=MainThread file=base_modinput.py:log_debug:286 | Next URL (@odata.nextLink): https://graph.microsoft.com/beta/auditLogs/signIns?$orderby=createdDateTime&$filter=createdDateTime+...
2019-04-25 10:27:56,178 DEBUG pid=6441 tid=MainThread file=connectionpool.py:_new_conn:809 | Starting new HTTPS connection (1): graph.microsoft.com

0 Karma

elle118
Engager

Hi @subbarayudu,

When I faced the same problem, I could improve the collection rate to over 99% by modifying the API calls that collect events (Sign-ins and Audit Logs) as follows.

ex) When the delay time is 5 minutes
$SPLUNK_HOME/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_audit.py

50     event_source = "tenant_id:%s" % tenant_id
51     query_date = get_start_date(helper, check_point_key)
52     query_date_end = (datetime.datetime.utcnow() - datetime.timedelta(minutes=5)).strftime('%Y-%m-%dT%H:%M:%S.%fZ')
53     access_token = azauth.get_access_token(client_id, client_secret, tenant_id)
54
55     if(access_token):
56         url = "https://graph.microsoft.com/beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+%s)+and+(activityDateTime+le+%s)" % (query_date, query_date_end)
57         audit_events = azutils.get_items(helper, access_token, url)

$SPLUNK_HOME/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_signins.py

52     event_source = "tenant_id:%s" % tenant_id
53     query_date = get_start_date(helper, check_point_key)
54     query_date_end = (datetime.datetime.utcnow() - datetime.timedelta(minutes=5)).strftime('%Y-%m-%dT%H:%M:%S.%fZ')
55     access_token = azauth.get_access_token(client_id, client_secret, tenant_id)
56
57     if(access_token):
58         url = "https://graph.microsoft.com/beta/auditLogs/signIns?$orderby=createdDateTime&$filter=(createdDateTime+gt+%s)+and+(createdDateTime+le+%s)" % (query_date, query_date_end)
59         sign_ins = azutils.get_items(helper, access_token, url)

There is no need to restart the Splunk service after fixing.
The corrected API call will be executed at the next collection timing, and events from the checkpoint up to 5 minutes before the acquisition timing will be collected.

If you want to check that the modified API call is being executed, you can check it from the DEBUG log of App.

ex) Sign-ins

2019-05-31 15:09:23,282 DEBUG pid=32155 tid=MainThread file=connectionpool.py:_make_request:400 | https://graph.microsoft.com:443 "GET /beta/auditLogs/signIns?$orderby=createdDateTime&$filter=(createdDateTime+gt+2019-05-31T05:58:52.8129242Z)+and+(createdDateTime+le+2019-05-31T06:04:22.013821Z) HTTP/1.1" 200 None

rkantamaneni_sp
Splunk Employee
Splunk Employee

Hi @subbarayudu,

The issue is most likely related to the MS API itself. As a test, if you could make a call to the API separately and download the logs locally, you can then ingest these into Splunk in a separate test_index and compare against the results you have.

This Splunk Answer thread explains the situation: https://answers.splunk.com/answers/719725/input-settings-for-microsoft-office-365-reporting.html

Here’s the "gotcha" though - Microsoft may delay message trace logs up to 24 hours. During this delay, message traces may come out of sequence. Continuing our example above, a message trace log with a time stamp of 1:29 PM may have come in delayed. If we are already requesting data from 1:30 PM to 2:30 PM, we willl miss this delayed event. The delay throttle makes sure we don’t go too fast and potentially miss events.

And it's also worth checking: https://answers.splunk.com/answers/731712/time-skew-for-when-logs-are-read.html#answer-733579

Hopefully this helps explain what you're seeing. If not, please do let us know.

0 Karma

subbarayudu
New Member

Hi rkantamaneni,

We do noticed splunk is unable to receive the complete logs from Azure Portal, Logs that are available in Azure Sigin logs are not forwarded to splunk, As part of validation, we searched for a user log in Azure portal and able to view activity in azure portal, but same log is not available in splunk, As part of troubleshooting we noticed during logs are being forwarded to splunk, but specific user log is not available in splunk, in splunk we even queried with Co-relation ID as well. Kindly help to address the issue.

Thanks,
Subbu

0 Karma

rkantamaneni_sp
Splunk Employee
Splunk Employee

Hi @subbarayudu ,

>> previous version app is unable to collect the complete sigin-in logs that is visible in Azure Portal.

Yes, the APIs in the newer version of the app were switched over to the MS Graph API so it matches what you see in the Azure Portal vs. before.

>> 2019-04-25 10:27:56,177 DEBUG

The messages you posted seem to be DEBUG level messages rather than ERROR messages. The DEBUG messages are from the App for whatever the App makers / developer(s) decide to print out to follow the internal actions of the App. The current messages don't seem to be representative of any kind of error, were these the right messages you meant to post?

@jconger can correct me if I missed anything.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...