Hey Guys
We are trying to configure Splunk with S3 and facing issues :
Have a few questions :
1) what should be under
Configure the remote volume
We have storageType:remote
what does [volume:s3] signify?
2) Do the entries below suffice ?
storageType = remote
path = s3://splunk-smartstore/indexes
remote.s3.supports_versioning = false
remote.s3.endpoint = http://<IP-address>/splunk-smartstore
remote.s3.access_key = <Access_key>
remote.s3.secret_key = <secrey key>
We keep seeing the following errors :
/opt/splunk/etc/master-apps/_cluster/local]# /opt/splunk/bin/./splunk cmd splunkd rfs -- ls
error: <remote_id> expected
error: operation failed; check log for details
What log file can help debugging this ?
Ok, what is the storage, it’s peak capacity, network capacity on nodes, middle and storage side?
Also your daily indexing volume and search profile?
Hi @anuragschandra ...
https://docs.splunk.com/Documentation/Splunk/8.2.1/admin/Indexesconf#indexes.conf.example
### This example demonstrates how to configure a volume that points to ### S3-based remote storage and indexes that use this volume. The setting ### "storageType=remote" indicates that this is a remote-storage volume. ### The "remotePath" parameter associates the index with that volume ### and configures a top-level location for uploading buckets. [volume:s3] storageType = remote path = s3://remote_volume remote.s3.bucket_name = example-s3-bucket remote.s3.access_key = S3_ACCESS_KEY remote.s3.secret_key = S3_SECRET_KEY
also pls check this
https://docs.splunk.com/Documentation/Splunk/8.2.1/Indexer/ConfigureremotestoreforSmartStore
also this page has got some good details on indexes.conf for S3:ac
https://blog.arcusdata.io/how-to-set-up-splunk-smart-store-in-aws
Still confused with what remote volume needs to have ?
Can somebody lay out step by step whats needed on the Storage side
Here is my assumption :
1) S3 bucket
2) Access ID
3) Secret Key
What does [volume:s3] signify ? is s3 a folder inside the s3 bucket ?
Also , what log file should we look at for failures?