COVID-19 response: Is Splunk able to ingest logs from Zoom?

[nick405060]

My Seattle-based company just onboarded Zoom for our rapid remote access expansion in response to COVID-19. We are looking to perform analytics on the logs immediately. I see there is an app pushing out from Splunk to Zoom, but cannot find any documentation how to ingest Zoom data. I do see that Zoom has an API.

Any assistance here?

[kruane]

Having some trouble here. I was using this to setup things up: https://www.youtube.com/watch?v=8pH_XRjPMjM

Steps Completed:

1.) Created Index

2.) I've add the Splunk Connect for Zoom

3.) Installed Splunk App for Zoom

For Zoom:

1.) I've gone into Zoom and now have an administrative account.

2.) Went to: https://marketplace.zoom.us/

3.) I completed the information about the setup, company name, username etc. 

4.) The trouble occurs on the last part where you enable the "Event subscriptions". It will not accept my url. Not exactly sure why. I tried various ways and it keeps saying "invalid url" 

[niketn]

@nick405060

Here is the Splunk App for Zoom (Splunk Works) and Splunk Connect for Zoom Addon
(Splunk) on Splunkbase!

[singhg]

Hello @niketnilay

We are trying to install Zoom Splunk Connect for Zoom add-on to pull data.

Question: can we use existing HTTP Event Collector (HEC) or we need to configure JSON Web Token (JWT) webhook input using the Add-on?

[Patrick_Peeters]

If you haven't implemented it yet (or for others finding this thread), have a look at either the Remote Work Insights dashboard (https://github.com/splunk/rwi_executive_dashboard) that Splunk recently published. That includes monitoring Zoom meetings, there's a step-by-step instruction guide in Zoom section (https://github.com/splunk/rwi_executive_dashboard/blob/master/RUNBOOK.md#zoom-walkthrough) on the RWI Github page expanding on the steps that @nick405060 described above.

Edit - updated links as they weren't visible in original post.

[nick405060]

Would it be possible to add Zoom IM metrics to RWI? I know that my CIO is not the only one interested in these metrics, and unless I'm mistaken these metrics aren't available anywhere even if you're doing direct API calls

(The only API call available is /chat/users/{userId}/messages, and that data cannot be pulled in bulk. I know I could write a Python script that queries the API for all users, one that then programmatically loops and queries the API IMs for each user, but this seems inelegant)

[Patrick_Peeters]

Looking at the Zoom API documentation there's a Chat Message Sent webhook that might be what you're after? https://marketplace.zoom.us/docs/api-reference/webhook-reference/chat-message-events/chat-message-se...

[nick405060 · Accepted solution]

Here's how:

• Get admin perms in Zoom, go to zoom.us, go to app marketplace, go to develop, click build app
• I did JWT, fill that out and copy the JWT token
• Download add-on builder from SplunkBase (I placed on indexer)
• Build app that takes Query URL and JWT Token, define REST URL as https://api.zoom.us/v2/${query_url}?access_token=${jwt_token}, and add access_token:${jwt_token} as a REST URL param
• Reboot indexer
• You'll have to write your own parsing regex. I just did some split, mvexpand, and basic rexing to get the data I need, but will write some more robust regex's soon

To get participant metrics you need to query the API for meetings IDs first, and then loop through and query the API for participant or QoS metrics for each meeting ID. I did this using a combination of Python and Splunk. Same thing with IM metrics, pull users first and then loop through for IMs.

Python script for participant metrics:

me@bin$ cat zoomparticipants.py
# Input field needs to be called "id" output field will be called "participant_string"

import requests
import splunk.Intersplunk
import sys

def generateResult(results, settings):
    for row in results:
        r = requests.get("https://api.zoom.us/v2/metrics/meetings/"+row['id']+"/participants?access_token=DELETED&type=past&page_size=300&page_number=")
        row['participant_string'] = r.json()
    return results

# Entry point of the code
results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()
results = generateResult(results, settings)
splunk.Intersplunk.outputResults(results)

SPL:

index=zoom source=zoom_api://metrics_meetings_page* | eval raw=split(_raw,"{\"uuid\":\"") | mvexpand raw | search raw!={* | rex field=raw "^(?<meeting_uuid>.*?)\"," | dedup meeting_uuid | 
eval id=meeting_uuid | zoomparticipants | eval participant=split(participant_string,"{") | mvexpand participant | rex field=participant "u\'id\': u\'(?<participant_id>.*?)\'" | where isnotnull(participant_id) | 
rex field=raw "\"id\":\"?(?<meeting_id>.*?)\"?," |
rex field=raw "\"topic\":\"?(?<meeting_topic>.*?)\"?," |
rex field=raw "\"host\":\"?(?<meeting_host>.*?)\"?," |
rex field=raw "\"email\":\"?(?<meeting_email>.*?)\"?," |
rex field=raw "\"user_type\":\"?(?<meeting_user_type>.*?)\"?," |
rex field=raw "\"start_time\":\"?(?<meeting_start_time>.*?)\"?," |
rex field=raw "\"end_time\":\"?(?<meeting_end_time>.*?)\"?," |
rex field=raw "\"duration\":\"?(?<meeting_duration>.*?)\"?," |
rex field=raw "\"participants\":\"?(?<meeting_participants>.*?)\"?," |
rex field=raw "\"has_pstn\":\"?(?<meeting_has_pstn>.*?)\"?," |
rex field=raw "\"has_voip\":\"?(?<meeting_has_voip>.*?)\"?," |
rex field=raw "\"has_3rd_party_audio\":\"?(?<meeting_has_3rd_party_audio>.*?)\"?," |
rex field=raw "\"has_video\":\"?(?<meeting_has_video>.*?)\"?," |
rex field=raw "\"has_screen_share\":\"?(?<meeting_has_screen_share>.*?)\"?," |
rex field=raw "\"has_recording\":\"?(?<meeting_has_recording>.*?)\"?," |
rex field=raw "\"has_sip\":\"?(?<meeting_has_sip>.*?)\"?," |
rex field=raw "\"dept\":\"?(?<meeting_dept>.*?)\"?}" |
rex field=participant "u\'speaker\': u\'(?<participant_speaker>.*?)\'" |
rex field=participant "u\'join_time\': u\'(?<participant_join_time>.*?)\'" |
rex field=participant "u\'mac_addr\': u\'(?<participant_mac_addr>.*?)\'" |
rex field=participant "u\'share_desktop\': u\'(?<participant_share_desktop>.*?)\'" |
rex field=participant "u\'device\': u\'(?<participant_device>.*?)\'" |
rex field=participant "u\'data_center\': u\'(?<participant_data_center>.*?)\'" |
rex field=participant "u\'domain\': u\'(?<participant_domain>.*?)\'" |
rex field=participant "u\'ip_address\': u\'(?<participant_ip_address>.*?)\'" |
rex field=participant "u\'share_application\': u\'(?<participant_share_application>.*?)\'" |
rex field=participant "u\'user_name\': u\'(?<participant_user_name>.*?)\'" |
rex field=participant "u\'harddisk_id \': u\'(?<participant_harddisk_id>.*?)\'" |
rex field=participant "u\'user_id\': u\'(?<participant_user_id>.*?)\'" |
rex field=participant "u\'location\': u\'(?<participant_location>.*?)\'" |
rex field=participant "u\'recording\': u\'(?<participant_recording>.*?)\'" |
rex field=participant "u\'share_whiteboard\': u\'(?<participant_share_whiteboard>.*?)\'" |
rex field=participant "u\'connection_type\': u\'(?<participant_connection_type>.*?)\'" |
rex field=participant "u\'network_type\': u\'(?<participant_network_type>.*?)\'" |
rex field=participant "u\'pc_name\': u\'(?<participant_pc_name>.*?)\'" |
rex field=participant "u\'microphone\': u\'(?<participant_microphone>.*?)\'" |
rex field=participant "u\'leave_time\': u\'(?<participant_leave_time>.*?)\'" |
rex field=participant "u\'leave_reason\': u\'(?<participant_leave_reason>.*?)\'" |
eval participant_seconds=strptime(participant_leave_time,"%Y-%m-%dT%H:%M:%SZ")-strptime(participant_join_time,"%Y-%m-%dT%H:%M:%SZ") |
table meeting_id meeting_topic meeting_host meeting_email meeting_user_type meeting_start_time meeting_end_time meeting_duration meeting_participants meeting_has_pstn meeting_has_voip meeting_has_3rd_party_audio meeting_has_video meeting_has_screen_share meeting_has_recording meeting_has_sip meeting_dept participant_speaker participant_join_time participant_mac_addr participant_share_desktop participant_device participant_data_center participant_domain participant_ip_address participant_share_application participant_user_name participant_harddisk_id participant_user_id participant_location participant_recording participant_share_whiteboard participant_connection_type participant_network_type participant_pc_name participant_microphone participant_leave_time participant_leave_reason participant_id participant_seconds

[nick405060]

PACs:

me@bin$ cat zoompacs.py
# Input field needs to be called "id" output field will be called "pac_string"

import requests
import splunk.Intersplunk
import sys

def generateResult(results, settings):
    for row in results:
        r = requests.get("https://api.zoom.us/v2/users/"+row['id']+"/pac?access_token=DELETED&type=past&page_size=300&page_number=")
        row['pac_string'] = r.json()
    return results

# Entry point of the code
results, dummyresults, settings = splunk.Intersplunk.getOrganizedResults()
results = generateResult(results, settings)
splunk.Intersplunk.outputResults(results)

[richardphung]

Here is the Zoom Log API Documentation:
https://marketplace.zoom.us/docs/api-reference/zoom-api/reports/reportoperationlogs

This will show you all of the consumption URLs.
e.g. daily logins: https://api.zoom.us/v2/report/daily

But pay attention to the query parameters (input) and the schema (output)
This tells you what output to expect, as these are different depending on what you are trying to query.

Then using the Splunk Add-on Builder, I would create a REST modular input:
https://docs.splunk.com/Documentation/AddonBuilder/3.0.1/UserGuide/ConfigureDataCollection

You can also populate identity lookups with something like:
https://api.zoom.us/v2/accounts/{accountId}/users

*You will need to log into your Zoom tenant and generate an OAUTH token to get started.

The add-on builder on Splunkbase: https://splunkbase.splunk.com/app/2962/

[richardphung]

Looks like there is a Splunk-to-Zoom notification webhook:
https://marketplace.zoom.us/apps/uSIGA6A2SKWmL9LrJ49BmQ?zcid=1231
https://zoomappdocs.docs.stoplight.io/splunk

But just short of using the Add-On-Builder to create your own RESTful Modular Input for Zoom.
https://splunkbase.splunk.com/app/2962/

I don't see a native Splunkbase App for this yet..