topic Re: Splunk Indexes data pointing to different storage in Splunk Enterprise

Splunk Indexes data pointing to different storage

elend — Thu, 31 Jul 2025 16:47:25 GMT

Hi there,

I want to point the secondary storage of my splunk indexer to mix with another storage, like point it to cloud storage?

so it will like this one is the common:

[volume:hot1]
path = /mnt/fast_disk
maxVolumeDataSizeMB = 100000

[volume:s3volume]
storageType = remote
path = s3://<bucketname>/rest/of/path

is there a mechanism or reference to did this?

Re: Splunk Indexes data pointing to different storage

richgalloway — Thu, 31 Jul 2025 18:04:21 GMT

You've just stumbled across SmartStore (S2). S2 keeps hot buckets local and copies warm buckets to S3. A cache of roughly 30 days of data is retained locally for faster search performance.

To implement S2 correctly, see https://docs.splunk.com/Documentation/Splunk/9.4.2/Indexer/AboutSmartStore

Re: Splunk Indexes data pointing to different storage

livehybrid — Thu, 31 Jul 2025 18:55:36 GMT

Hi @elend

Yes, you can configure Splunk (since 7.2 I think) to use mixture of local storage and S3-compliant storage, including the likes of Amazon S3 using Splunk's SmartStore functionality, this essentially uses your local storage for hot buckets and as a local cache for buckets which are also stored in S3. Its more of a complex beast than I can go into here, and there are lots of things to consider - for example this is generally considered a one-way exercise!

https://docs.splunk.com/Documentation/SVA/current/Architectures/SmartStore gives a good overview of the architecture, benefits and next steps.

Check out https://help.splunk.com/en/splunk-enterprise/administer/manage-indexers-and-indexer-clusters/9.3/manage-smartstore/configure-smartstore for more info on setting up smartstore as well as https://help.splunk.com/en/splunk-enterprise/administer/manage-indexers-and-indexer-clusters/9.4/deploy-smartstore/deploy-smartstore-on-a-new-standalone-indexer which has some info on setting this up on a single indexer (as a starter, this will depend on your specific environment architecture).

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Splunk Indexes data pointing to different storage

PrewinThomas — Fri, 01 Aug 2025 04:55:30 GMT

@elend

As @richgalloway mentioned,
This is exactly what SmartStore is designed for.

Hot buckets stay on local disk for fast ingestion and search and warm buckets are offloaded to remote storage (e.g., S3).

#https://docs.splunk.com/Documentation/SVA/current/Architectures/SmartStore

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!

Re: Splunk Indexes data pointing to different storage

elend — Fri, 01 Aug 2025 06:19:32 GMT

Oh thankyou. so thats just point the bucket like the sample right?

Re: Splunk Indexes data pointing to different storage

livehybrid — Fri, 01 Aug 2025 07:38:00 GMT

Hi @elend

You configure a volume in your indexes.conf which is your s3 location essentially, and then you can update all or individual indexes to use that volume by setting the remotePath eg

remotePath = volume:<VOLUME_NAME>/$_index_name

the $_index_name is actually an internal variable so you don’t need to overwrite this.
in addition to the other docs I posted on the previous post it’s worth checking https://docs.splunk.com/Documentation/Splunk/9.4.2/Indexer/SmartStoresecuritystrategies too.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful

Marking it as the solution if it resolved your issue

Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing.