All Apps and Add-ons

Splunk smart store S3 bucket config to indexes 7.2.x

Splunk_citizen
Explorer

Hello Splunkers,

Any one had luck to test for connective from indexer host to send data to your AWS S3 one ? I have tried with below settings to setup frozen volume but its giving error.

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

Tags (4)
0 Karma

myriadic
Path Finder

you shouldn't have "s3/" in front of "frozen" in volume:s3/frozen

also, remotePath needs an index name

0 Karma

Splunk_citizen
Explorer

Thank you but i have tried below ways

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

but no luck ...

0 Karma

myriadic
Path Finder

if you have "remotePath = volume:s3/frozen" under [default], your "index1" is going to have:

remotePath = volume:s3/frozen

that makes no sense.

replace "remotePath = volume:s3/frozen" with "remotePath = volume:frozen/$_index_name"

0 Karma

Splunk_citizen
Explorer

Thanks i have tried all approaches if you have working settings please post it here.

Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath

0 Karma

myriadic
Path Finder

[default]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

[os]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

0 Karma

Splunk_citizen
Explorer

Thank you for your quick reply but how data will be written in remote path based on ? what condition ?
I just want to create my new index and all data to be in only S3 bucket hot warm and frozen data or else just want to send frozen data to remote path

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...