topic forwarding logs to a gcp bucket in Getting Data In

forwarding logs to a gcp bucket

yvan-rostand — Wed, 10 Jan 2024 11:03:36 GMT

Hi,

I am trying to to forward logs from a heavy forwarder to a gcp bucket using the outputs.conf, but it has been unsuccessful (no logs seen in the bucket). Not sure if that has to do with my config file or something else.

Can anyone help me with an example?

This is my outputs.conf and I don't know what is wrong.

# BASE SETTINGS [tcpout] defaultGroup = primary_indexers forceTimebasedAutoLB = true [tcpout:bucket_index] indexAndForward = true forwardedindex.0.whitelist = my_index [bucket] compressed = false json_escaping = auto google_storage_key = “12345abcde” google_storage_bucket = my-gcp-bucket path = /path/my-gcp-bucket route = bucket_index

Re: forwarding logs to a gcp bucket

richgalloway — Tue, 09 Jan 2024 19:15:39 GMT

The problem is Splunk can't do that. An HF can forward to another Splunk instance or to a syslog receiver. They cannot send directly to a storage device/service.

Re: forwarding logs to a gcp bucket

inventsekar — Wed, 10 Jan 2024 01:00:13 GMT

Hi @yvan-rostand

As per my understanding, you should use props and transform.conf as well.

Maybe could you pls try this idea - forward data to 3rd party systems:

https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Forwarddatatothird-partysystemsd