Deployment Architecture

Splunk smart store S3 config indexes to send data to remote location

Splunk_citizen
Explorer

Hello Splunkers,

i was just doing some POC to send data from our on prem Splunk indexer to AWS s3 bucket as they added new features in 7.2.X

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY

But still i did not see any data written on remote location any thoughts ?

woodcock
Esteemed Legend

Leave specific feedback on the docs page. They will get back to you quickly with answers.

0 Karma

p_gurav
Champion

Can you try this :

[volume:s3]
storageType = remote
path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYY

[index1]
remotePath = volume:s3/$_index_name
0 Karma

Splunk_citizen
Explorer

got this error Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

0 Karma

harsmarvania57
Ultra Champion

This was recently discussed on Slack (indexer-clustering channel) and configuration for index should be like this

[index1]
remotePath = volume:s3/$_index_name
homePath = $SPLUNK_DB/$_index_name/db
coldPath = $SPLUNK_DB/$_index_name/colddb
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
0 Karma

cpharvey
Explorer

typo in voLume ?

0 Karma

harsmarvania57
Ultra Champion

Yes, corrected it. Thanks 🙂

0 Karma

Splunk_citizen
Explorer

I have tried below one

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

keep on getting below error
Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If y

0 Karma
Get Updates on the Splunk Community!

Developer Spotlight with Brett Adams

In our third Spotlight feature, we're excited to shine a light on Brett—a Splunk consultant, innovative ...

Index This | What can you do to make 55,555 equal 500?

April 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Say goodbye to manually analyzing phishing and malware threats with Splunk Attack ...

In today’s evolving threat landscape, we understand you’re constantly bombarded with phishing and malware ...