Getting Data In

How to Integrate gmail logs into splunk?

Arch
Loves-to-Learn Lots

Hi,

We are trying to integrate gmail logs into our Splunk Cloud instance.  We have tried the 'Splunk Addon for Google Workspace(https://splunkbase.splunk.com/app/5556/)'. The integration was smooth, and we were able to see gsuite header logs in Splunk. But the problem in this case was it eventually generated large bills from Google for the bigqueries. Hence we were forced to disable it temporarily. When we did an analysis, we found that the current approach in this addon is to query all partition at once using the below query:
"SELECT * FROM `{gcp_project_id}.gmail_logs_dataset.daily_*` "
                    "WHERE event_info.timestamp_usec > {start_time_usec} "
                    "AND event_info.timestamp_usec < {end_time_usec} "
                    "ORDER BY event_info.timestamp_usec ASC"

Instead of querying the whole partition, we would like to query the table of each day, it would massively reduce the cost. 

I did raise a support ticket with Splunk on this, and they have confirmed it requires a code change and they cannot assure any timeline for this. Even though we tried to manually edit this part of code and upload it via custom app, it didnt succeed the vetting process. 

It would be really helpful if someone could provide me an alternate solution for integrating gmail logs or a way to upload the modified addon.

Much appreciated,

Archa

Labels (1)
Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

I think you had the right idea to change the app yourself rather than waiting for Splunk to do it.

Please tell us why your customization failed the vetting process.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Arch
Loves-to-Learn Lots

Hi,

Thank you for the suggestion. I was able to pass the vetting process. But when I try to install the custom app, I'm getting the below error.

Screenshot 2022-08-24 at 11.07.57.png

 Please let me know how can I fix this issue

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That error message is unhelpful.  It could be a transient error or not.  Contact Splunk Support if it continues.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...