Getting Data In

How to Integrate gmail logs into splunk?

Arch
Loves-to-Learn Lots

Hi,

We are trying to integrate gmail logs into our Splunk Cloud instance.  We have tried the 'Splunk Addon for Google Workspace(https://splunkbase.splunk.com/app/5556/)'. The integration was smooth, and we were able to see gsuite header logs in Splunk. But the problem in this case was it eventually generated large bills from Google for the bigqueries. Hence we were forced to disable it temporarily. When we did an analysis, we found that the current approach in this addon is to query all partition at once using the below query:
"SELECT * FROM `{gcp_project_id}.gmail_logs_dataset.daily_*` "
                    "WHERE event_info.timestamp_usec > {start_time_usec} "
                    "AND event_info.timestamp_usec < {end_time_usec} "
                    "ORDER BY event_info.timestamp_usec ASC"

Instead of querying the whole partition, we would like to query the table of each day, it would massively reduce the cost. 

I did raise a support ticket with Splunk on this, and they have confirmed it requires a code change and they cannot assure any timeline for this. Even though we tried to manually edit this part of code and upload it via custom app, it didnt succeed the vetting process. 

It would be really helpful if someone could provide me an alternate solution for integrating gmail logs or a way to upload the modified addon.

Much appreciated,

Archa

Labels (1)
Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

I think you had the right idea to change the app yourself rather than waiting for Splunk to do it.

Please tell us why your customization failed the vetting process.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Arch
Loves-to-Learn Lots

Hi,

Thank you for the suggestion. I was able to pass the vetting process. But when I try to install the custom app, I'm getting the below error.

Screenshot 2022-08-24 at 11.07.57.png

 Please let me know how can I fix this issue

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That error message is unhelpful.  It could be a transient error or not.  Contact Splunk Support if it continues.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...