How to get logs from Azure and O365 into Splunk?

[marycordova]

Problem: various apps and TAs exist but none of them are reliable and/or supported.

[marycordova]

[marycordova]

[pmeyerson]

Do you have any idea on which (if any) subscriptions this feature is included in? I'm having a tough time understanding how all the different o365+azure -> splunk options are priced from the msft side.
Wasn't sure if you uncovered anything while looking into this option.

[marycordova]

I think you need like an "E3" pricing tier but I'm really not sure...

[teddyidc1101]

Do you have solution for Skype?

[marycordova]

For Skype, even though the logs are visible in the same portal.office.com place as all the other O365 logs they have not yet added them to the Azure integration. So right now you'd have to write a powershell script or something to grab them, probably from the API...which I hate cuz I've never met an API based app that didn't break, but give me something like syslog, or hec...never had one that did break!

[marycordova]

[marycordova · Accepted solution]

see additional screenshots in below comments as I can't post them all in this answer

Splunk:
setup a Splunk RAW http(s) endpoint for Azure and/or O365 (must be raw not regular hec or timestamping is all messed up)

inputs.conf:
[http://inputs_azure]
disabled = 0
index = azure
sourcetype = httpevent:azure
token = token

[http://inputs_o365]
disabled = 0
index = azure
sourcetype = httpevent:o365
token = token

props.conf:
[httpevent:azure]
KV_MODE = json
TIME_PREFIX = ^\D+
TIME_FORMAT = %Y-%m-%dT%H:%M:%SZ
TZ = UTC

[httpevent:o365]
KV_MODE = json
TIME_PREFIX = ^\D+
TIME_FORMAT = %Y-%m-%dT%H:%M:%SZ
TZ = UTC

Microsoft: You need some “solutions”
- O365 "solution": Office 365 Analytics (Preview)
- https://azuremarketplace.microsoft.com/en-us/marketplace/apps/Microsoft.Office365OMS?tab=Overview
- Azure audit "solution": Activity Log Analytics
- https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-activity

1. in the back end a “log analytics” repo for the logs ingested by each solution is created
2. create a “logic app” for each repo that will query log analytics directly and post http(s) to the Splunk RAW endpoint
3. set query backward in time (I have a 5 hour delay but I think that could be shortened to 2 hours) because MS doesn’t deliver logs to the solution/log analytics in real time
4. Only outstanding issue is that super nested json isn’t parsing…

[marycordova]

damn...what happened to those screenshots? there is literally no way i will ever be able to re-create them since this is $job-1