Getting Data In

How to Integrate gmail logs into splunk?

Arch
Loves-to-Learn Lots

Hi,

We are trying to integrate gmail logs into our Splunk Cloud instance.  We have tried the 'Splunk Addon for Google Workspace(https://splunkbase.splunk.com/app/5556/)'. The integration was smooth, and we were able to see gsuite header logs in Splunk. But the problem in this case was it eventually generated large bills from Google for the bigqueries. Hence we were forced to disable it temporarily. When we did an analysis, we found that the current approach in this addon is to query all partition at once using the below query:
"SELECT * FROM `{gcp_project_id}.gmail_logs_dataset.daily_*` "
                    "WHERE event_info.timestamp_usec > {start_time_usec} "
                    "AND event_info.timestamp_usec < {end_time_usec} "
                    "ORDER BY event_info.timestamp_usec ASC"

Instead of querying the whole partition, we would like to query the table of each day, it would massively reduce the cost. 

I did raise a support ticket with Splunk on this, and they have confirmed it requires a code change and they cannot assure any timeline for this. Even though we tried to manually edit this part of code and upload it via custom app, it didnt succeed the vetting process. 

It would be really helpful if someone could provide me an alternate solution for integrating gmail logs or a way to upload the modified addon.

Much appreciated,

Archa

Labels (1)
Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

I think you had the right idea to change the app yourself rather than waiting for Splunk to do it.

Please tell us why your customization failed the vetting process.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Arch
Loves-to-Learn Lots

Hi,

Thank you for the suggestion. I was able to pass the vetting process. But when I try to install the custom app, I'm getting the below error.

Screenshot 2022-08-24 at 11.07.57.png

 Please let me know how can I fix this issue

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That error message is unhelpful.  It could be a transient error or not.  Contact Splunk Support if it continues.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...