All Apps and Add-ons

has anyone successful setup the remotePath option in indexes.conf in Splunk 7.0 to work with indexed data in s3?

Log_wrangler
Builder

I was trying to search copies of indexed data in S3.

Has anyone had luck with this scenario using remotePath??? I know it says not supported but is it functional at this point?

indexes.conf for splunk 7.0
[https://docs.splunk.com/Documentation/Splunk/7.0.1/Admin/Indexesconf]

remotePath =
* Currently not supported. This setting is related to a feature that is
still under development.
* Optional.
* Presence of this parameter means that this index uses remote storage, instead
of the local file system, as the main repository for bucket storage. The
index processor works with a cache manager to fetch buckets locally, as
necessary, for searching and to evict them from local storage as space fills
up and they are no longer needed for searching.
* This setting must be defined in terms of a storageType=remote volume
definition. See the volume section below.
* The path portion that follows the volume reference is relative to the path
specified for the volume. For example, if the path for a volume "v1" is
"s3://bucket/path" and "remotePath" is "volume:v1/idx1", then the fully
qualified path will be "s3://bucket/path/idx1". The rules for resolving the
relative path with the absolute path specified in the volume can vary
depending on the underlying storage type.
* If "remotePath" is specified, the "coldPath" and "thawedPath" attributes are
ignored. However, they still must be specified.

Any advise or lessons learned is appreciated.

Thank you

0 Karma

nickhills
Ultra Champion
remotePath = <root path for remote volume, prefixed by a URI-like scheme>

Currently not supported. This setting is related to a feature that is
still under development.

Even if i knew, I couldn't tell you!
:)

My guess is that its to allow for cloud storage like S3 - hopefully for archive/frozen data - but thats entirely a guess
But i am watching the releases very closely for more info

If my comment helps, please give it a thumbs up!

nickhills
Ultra Champion

Just a follow up comment - I very much doubt that this is going to allow you to store hot/warm or even cold data in s3 - the read performance isn't up to the job, and the write mechanism would be far too clunky for indexers.

If my comment helps, please give it a thumbs up!
0 Karma