Best Approach for Integrating Google Workspace with Splunk

[vnetrebko]

Hello Splunk Community,

We are currently evaluating different options for integrating Google Workspace with Splunk and would appreciate your guidance on which approach might be the most effective and sustainable.

The options we are considering are:

1. Splunk App for Google Workspace (App 5498)

2. Splunk App for Google Workspace (App 5556)

3. Integration via GCP BigQuery → Pub/Sub → Splunk

We initially tested the first option (App 5498), but we encountered limitations related to the number of API requests, which makes this approach less convenient for our needs. Because of this, we are exploring the other options.

Has anyone had experience comparing these approaches? Which method would you recommend in terms of scalability, performance, and long-term maintainability?

Thank you in advance for your insights.

[Meett]

Hello @vnetrebko ,

I'd like to provide insights into the alternative options you're exploring.

Option 1: Splunk App for Google Workspace (App 5556)

The Splunk App for Google Workspace (App 5556) is a comprehensive solution designed to collect Google Workspace event data using Google Workspace APIs. It provides CIM-compatible knowledge for use with other Splunk apps. Recent updates include:

• Version 2.8.0: Added new sourcetypes: gws:reports:chat, gws:reports:mobile, and gws:reports:chrome.

• Version 2.8.1: Fixed issues with gmail_logs_migrated input.

This app allows for structured data collection and is suitable for organizations looking to integrate Google Workspace data into Splunk for analysis and monitoring.

Option 2: Integration via GCP BigQuery → Pub/Sub → Splunk

For a more scalable and flexible integration, consider using Google Cloud Platform's (GCP) services:

1. BigQuery: Store and analyze large datasets.

2. Pub/Sub: Stream data in real-time.

3. Dataflow: Process and transform data.

4. Splunk HTTP Event Collector (HEC): Ingest data into Splunk.

This approach is beneficial for handling high volumes of data and offers greater control over data processing. Google Cloud's Dataflow template facilitates exporting data from Pub/Sub to Splunk, enabling the ingestion of various message types into Splunk.

Given your experience with App 5498 and its limitations, Option 2 (GCP BigQuery → Pub/Sub → Splunk) offers a more scalable and flexible solution. While it requires a more complex setup, it provides greater control over data processing and can handle higher data volumes effectively.

If this helps you please add it as an solution.

[vnetrebko]

UPD.
We used Splunk App for Google Workspace (App 5556)

[alacercogitatus]

Hello! Author of 5498 here! 

If you would please elaborate, what do you mean by "encountered limitations related to the number of API requests" ? Which sourcetypes? which inputs? What are the interval settings? Please let me know so I can continue to improve the product. Thank you!