All Apps and Add-ons

Splunk smart store S3 bucket config to indexes 7.2.x

Splunk_citizen
Explorer

Hello Splunkers,

Any one had luck to test for connective from indexer host to send data to your AWS S3 one ? I have tried with below settings to setup frozen volume but its giving error.

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

Tags (4)
0 Karma

myriadic
Path Finder

you shouldn't have "s3/" in front of "frozen" in volume:s3/frozen

also, remotePath needs an index name

0 Karma

Splunk_citizen
Explorer

Thank you but i have tried below ways

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY
[default]
remotePath = volume:s3/frozen

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

but no luck ...

0 Karma

myriadic
Path Finder

if you have "remotePath = volume:s3/frozen" under [default], your "index1" is going to have:

remotePath = volume:s3/frozen

that makes no sense.

replace "remotePath = volume:s3/frozen" with "remotePath = volume:frozen/$_index_name"

0 Karma

Splunk_citizen
Explorer

Thanks i have tried all approaches if you have working settings please post it here.

Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath

0 Karma

myriadic
Path Finder

[default]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

[os]
homePath = $SPLUNK_DB/$_index_name/db
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
coldPath = $SPLUNK_DB/$_index_name/colddb
remotePath = volume:remote_store/$_index_name

0 Karma

Splunk_citizen
Explorer

Thank you for your quick reply but how data will be written in remote path based on ? what condition ?
I just want to create my new index and all data to be in only S3 bucket hot warm and frozen data or else just want to send frozen data to remote path

0 Karma
Get Updates on the Splunk Community!

How to Get Started with Splunk Data Management Pipeline Builders (Edge Processor & ...

If you want to gain full control over your growing data volumes, check out Splunk’s Data Management pipeline ...

Out of the Box to Up And Running - Streamlined Observability for Your Cloud ...

  Tech Talk Streamlined Observability for Your Cloud Environment Register    Out of the Box to Up And Running ...

Splunk Smartness with Brandon Sternfield | Episode 3

Hello and welcome to another episode of "Splunk Smartness," the interview series where we explore the power of ...