I have set up a Cloud Storage Bucket input using the Splunk Add-on for Google Cloud Platform.
I do not see a way to easily configure the sourcetype. I added a sourcetype line in google_cloud_storage_buckets.conf, but when restarting Splunk it showed "Invalid key in stanza [______] in /opt/splunk/etc/apps/Splunk_TA_google-cloudplatform/local/google_cloud_storage_buckets.conf, line 8: sourcetype (value: ______)"
The only way I was able to get my desired sourcetype set was to edit the python script which is setting it (which is not a long-term solution as I may need to add more inputs of different sourcetypes later)
Even then, after updating the script to set the desired sourcetype, the config I set up for this sourcetype (regarding event breaking) is not working - even though it does work properly when adding the data from a local copy of the file in the GCP bucket. Is there anything the script is doing to the files from the bucket that is mucking with their format.... or a reason why the settings in props.conf for the sourcetype are not being applied to the files being pulled down from the GCP bucket?
@gn694 there is no sourcetype setting in add-on it supports only following storage bucket conf, Have you tried configuring them and test it. I assume modinputs python script will allocate one of the sourcetype supported by add-on.
google:gcp:buckets:csvdata -> CSV contents of objects present in the Cloud Storage Bucket
google:gcp:buckets:jsondata -> JSON contents of objects present in the Cloud Storage Bucket
google:gcp:buckets:xmldata -> XML contents of objects present in the Cloud Storage Bucket
google:gcp:buckets:data -> Generic source type for the contents of other file extensions. For example, txt, avro, and parquet None
google_credentials_name = stanza name in google_credentials.conf
google_project = the project name
bucket_name = bucket name
An upvote would be appreciated and Accept solution if it helps!