S3 indexes.conf issues

[anuragschandra]

Hey Guys

We are trying to configure Splunk with S3 and facing issues : 

Have a few questions :

1) what should be under 

Configure the remote volume
We have storageType:remote 

what does [volume:s3] signify? 

2) Do the entries below suffice ?

storageType = remote
path = s3://splunk-smartstore/indexes
remote.s3.supports_versioning = false
remote.s3.endpoint = http://<IP-address>/splunk-smartstore
remote.s3.access_key = <Access_key>
remote.s3.secret_key = <secrey key>

We keep seeing the following errors :

/opt/splunk/etc/master-apps/_cluster/local]# /opt/splunk/bin/./splunk cmd splunkd rfs -- ls
error: <remote_id> expected
error: operation failed; check log for details

What log file can help debugging this ?

[isoutamo]

What is your environment: AWS, onprem or mixed or Azure? If onprem, what are your S3 storage, nodes, networks etc?
R. Ismo

[anuragschandra]

This is on prem and S3 compatible storage.

[isoutamo]

Ok, what is the storage, it’s peak capacity, network capacity on nodes, middle and storage side?

Also your daily indexing volume and search profile?

[inventsekar]

Hi @anuragschandra ... 

https://docs.splunk.com/Documentation/Splunk/8.2.1/admin/Indexesconf#indexes.conf.example

### This example demonstrates how to configure a volume that points to
### S3-based remote storage and indexes that use this volume.  The setting
### "storageType=remote" indicates that this is a remote-storage volume.
### The "remotePath" parameter associates the index with that volume
### and configures a top-level location for uploading buckets.

[volume:s3]
storageType = remote
path = s3://remote_volume
remote.s3.bucket_name = example-s3-bucket
remote.s3.access_key = S3_ACCESS_KEY
remote.s3.secret_key = S3_SECRET_KEY

also pls check this 

https://docs.splunk.com/Documentation/Splunk/8.2.1/Indexer/ConfigureremotestoreforSmartStore

also this page has got some good details on indexes.conf for S3:ac

https://blog.arcusdata.io/how-to-set-up-splunk-smart-store-in-aws

[anuragschandra]

Still confused with what remote volume needs to have ?

Can somebody lay out step by step whats needed on the Storage side 

Here is my assumption :

1) S3 bucket 

2) Access ID

3) Secret Key 

What does [volume:s3] signify ? is s3 a folder inside the s3 bucket ?

Also , what log file should we look at for failures?