Deployment Architecture

Splunk smart store S3 config indexes to send data to remote location

Splunk_citizen
Explorer

Hello Splunkers,

i was just doing some POC to send data from our on prem Splunk indexer to AWS s3 bucket as they added new features in 7.2.X

[volume:frozen]
storageType = remote

path = s3://example-s3-bucket/remote_volume

path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYYY

But still i did not see any data written on remote location any thoughts ?

woodcock
Esteemed Legend

Leave specific feedback on the docs page. They will get back to you quickly with answers.

0 Karma

p_gurav
Champion

Can you try this :

[volume:s3]
storageType = remote
path = s3://xyz-splunk-bb/frozen
remote.s3.access_key = XXX
remote.s3.secret_key = YYY

[index1]
remotePath = volume:s3/$_index_name
0 Karma

Splunk_citizen
Explorer

got this error Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

0 Karma

harsmarvania57
Ultra Champion

This was recently discussed on Slack (indexer-clustering channel) and configuration for index should be like this

[index1]
remotePath = volume:s3/$_index_name
homePath = $SPLUNK_DB/$_index_name/db
coldPath = $SPLUNK_DB/$_index_name/colddb
thawedPath = $SPLUNK_DB/$_index_name/thaweddb
0 Karma

cpharvey
Explorer

typo in voLume ?

0 Karma

harsmarvania57
Ultra Champion

Yes, corrected it. Thanks 🙂

0 Karma

Splunk_citizen
Explorer

I have tried below one

[index1]
homePath = volume:frozen/index1/db
coldPath = volume:frozen/index1/colddb
thawedPath = volume:frozen/index1/thaweddb

keep on getting below error
Problem parsing indexes.conf: Cannot load IndexConfig: idx=index1 param=homePath Remote volume path specification is only valid for parameter remotePath
Validating databases (splunkd validatedb) failed with code '1'. If y

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...

Customer success is front and center at .conf25

Hi Splunkers, If you are not able to be at .conf25 in person, you can still learn about all the latest news ...