Deployment Architecture

Splunk smart store S3 config indexes to send data to remote location

Splunk_citizen
Explorer

Hello Splunkers,

i was just doing some POC to send data from our on prem Splunk indexer to AWS s3 bucket as they added new features in 7.2.X

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY

But still i did not see any data written on remote location any thoughts ?

woodcock
Esteemed Legend

Leave specific feedback on the docs page. They will get back to you quickly with answers.

0 Karma

p_gurav
Champion

Can you try this :

[volume:s3]
storageType = remote
path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYY

[index1]
remotePath = volume:s3/$_index_name
0 Karma

Splunk_citizen
Explorer

got this error Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

0 Karma

harsmarvania57
Ultra Champion

This was recently discussed on Slack (indexer-clustering channel) and configuration for index should be like this

[index1]
remotePath = volume:s3/$_index_name
homePath = $SPLUNK_DB/$_index_name/db
coldPath = $SPLUNK_DB/$_index_name/colddb
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
0 Karma

cpharvey
Explorer

typo in voLume ?

0 Karma

harsmarvania57
Ultra Champion

Yes, corrected it. Thanks 🙂

0 Karma

Splunk_citizen
Explorer

I have tried below one

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

keep on getting below error
Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If y

0 Karma
Get Updates on the Splunk Community!

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...

Purpose in Action: How Splunk Is Helping Power an Inclusive Future for All

At Cisco, purpose isn’t a tagline—it’s a commitment. Cisco’s FY25 Purpose Report outlines how the company is ...

[Upcoming Webinar] Demo Day: Transforming IT Operations with Splunk

Join us for a live Demo Day at the Cisco Store on January 21st 10:00am - 11:00am PST In the fast-paced world ...