Deployment Architecture

Splunk smart store S3 config indexes to send data to remote location

Splunk_citizen
Explorer

Hello Splunkers,

i was just doing some POC to send data from our on prem Splunk indexer to AWS s3 bucket as they added new features in 7.2.X

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY

But still i did not see any data written on remote location any thoughts ?

woodcock
Esteemed Legend

Leave specific feedback on the docs page. They will get back to you quickly with answers.

0 Karma

p_gurav
Champion

Can you try this :

[volume:s3]
storageType = remote
path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYY

[index1]
remotePath = volume:s3/$_index_name
0 Karma

Splunk_citizen
Explorer

got this error Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

0 Karma

harsmarvania57
Ultra Champion

This was recently discussed on Slack (indexer-clustering channel) and configuration for index should be like this

[index1]
remotePath = volume:s3/$_index_name
homePath = $SPLUNK_DB/$_index_name/db
coldPath = $SPLUNK_DB/$_index_name/colddb
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
0 Karma

cpharvey
Explorer

typo in voLume ?

0 Karma

harsmarvania57
Ultra Champion

Yes, corrected it. Thanks 🙂

0 Karma

Splunk_citizen
Explorer

I have tried below one

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

keep on getting below error
Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If y

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...