All Apps and Add-ons

Splunk smart store S3 bucket config to indexes 7.2.x

Splunk_citizen
Explorer

Hello Splunkers,

Any one had luck to test for connective from indexer host to send data to your AWS S3 one ? I have tried with below settings to setup frozen volume but its giving error.

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

Tags (4)
0 Karma

myriadic
Path Finder

you shouldn't have "s3/" in front of "frozen" in volume:s3/frozen

also, remotePath needs an index name

0 Karma

Splunk_citizen
Explorer

Thank you but i have tried below ways

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

but no luck ...

0 Karma

myriadic
Path Finder

if you have "remotePath = volume:s3/frozen" under [default], your "index1" is going to have:

remotePath = volume:s3/frozen

that makes no sense.

replace "remotePath = volume:s3/frozen" with "remotePath = volume:frozen/$_index_name"

0 Karma

Splunk_citizen
Explorer

Thanks i have tried all approaches if you have working settings please post it here.

Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath

0 Karma

myriadic
Path Finder

[default]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

[os]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

0 Karma

Splunk_citizen
Explorer

Thank you for your quick reply but how data will be written in remote path based on ? what condition ?
I just want to create my new index and all data to be in only S3 bucket hot warm and frozen data or else just want to send frozen data to remote path

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...