All Apps and Add-ons

Splunk smart store S3 bucket config to indexes 7.2.x

Splunk_citizen
Explorer

Hello Splunkers,

Any one had luck to test for connective from indexer host to send data to your AWS S3 one ? I have tried with below settings to setup frozen volume but its giving error.

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

Tags (4)
0 Karma

myriadic
Path Finder

you shouldn't have "s3/" in front of "frozen" in volume:s3/frozen

also, remotePath needs an index name

0 Karma

Splunk_citizen
Explorer

Thank you but i have tried below ways

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

but no luck ...

0 Karma

myriadic
Path Finder

if you have "remotePath = volume:s3/frozen" under [default], your "index1" is going to have:

remotePath = volume:s3/frozen

that makes no sense.

replace "remotePath = volume:s3/frozen" with "remotePath = volume:frozen/$_index_name"

0 Karma

Splunk_citizen
Explorer

Thanks i have tried all approaches if you have working settings please post it here.

Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath

0 Karma

myriadic
Path Finder

[default]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

[os]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

0 Karma

Splunk_citizen
Explorer

Thank you for your quick reply but how data will be written in remote path based on ? what condition ?
I just want to create my new index and all data to be in only S3 bucket hot warm and frozen data or else just want to send frozen data to remote path

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.