Getting Data In

GCS input with Google Cloud App

devopsbrightins
Engager

Hey there!

We're using Google Cloud App to inject logs sent to GCS by Stackdriver.

Stackdriver produces logs in GCS in JSON format with multiple events per file, newline separated. Each resulting json can be up to 200MB in size making Splunk input to choke on data.

Example (truncated for brevity):

gs://mybucket/stackdriver-logs/20210309.json:

{"insertId":"c4fc7617-638d-4553-a7c1-861b44b06299","labels":"blah"}
{"insertId":"6c386a11-ebed-42e0-9ceb-6db36c8ea40e","labels":{"blah blah"}

Can we configure Cloud App plugin or Splunk to  split each json document from file into its own event?

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...