topic Re: Relocate KVstore files on filesystem in Knowledge Management

Relocate KVstore files on filesystem

jimodonald — Thu, 07 Apr 2016 14:32:32 GMT

I am extensively using KV Store for a project and have run into an issue with where the files are stored on disk. There is a long sorted history to the disk layout, but I am not delving into that at this time.

The server is a Windows 2012 server running Splunk 6.3.3. The C: drive is 96GB in size (with 9.7GB free) and the F: drive is 2 TB in size (with 1.7 TB free). Splunk is installed to the C: drive as per corporate standards. My KV store is currently 60GB in size. My issue is that the KV store files exist in the default index location at $SPLUNK_HOME$\var\lib\splunk and I do not see any documentation related to relocating the KV store to a different filesystem like is possible with indexes.

I've tried to update the indexes.conf to point it to a different filesystem as follows:

[default]

[kvstore]
homePath   = F:\Splunk\var\lib\splunk\kvstore\db
coldPath   = F:\Splunk\var\lib\splunk\kvstore\colddb
thawedPath = $SPLUNK_DB\kvstore\thaweddb

However that does not move the mongo directory or the dumps directory under the kvstore directory.

I suspect (but have not tried yet) that updating the kvstore stanza to something like:

[kvstore]
mongoPath = F:\Splunk\var\lib\splunk\kvstore\mongo
dumpsPath = F:\Splunk\var\lib\splunk\kvstore\dumps

might to the trick, but thought I'd check with the larger audience before I go breaking stuff.

So how about it? Has anyone successfully relocated the kv store files by using indexes.conf? If so, how did you do it?

Thanks in advance.

Re: Relocate KVstore files on filesystem

DavidHourani — Fri, 07 Oct 2016 12:08:05 GMT

hello did you find any solution for that ?

Re: Relocate KVstore files on filesystem

jimodonald — Fri, 07 Oct 2016 13:10:43 GMT

I have not tested this yet -- haven't had time yet, but I did find this stanza in server.conf:

[kvstore]
dbPath = <path>
 * Path where KV Store data is stored.
 * Changing this directory after initial startup does not move existing data.
  The contents of the directory should be manually moved to the new
  location.
 * Defaults to $SPLUNK_DB/kvstore.

so in theory you should be able to define the path for the kvstore in server.conf. If you try it out, please post back with your results.

UPDATE: This has been tested successfully and worked exactly like you'd expect it to. Just like moving an index.

Re: Relocate KVstore files on filesystem

marcoscala — Sat, 10 Dec 2016 09:11:06 GMT

Good question! And what about relocating a single Collection inside the KVStore? I'm generating a sample collection for an app and I'd like to export just that collection instead of all the KV Store.
Is it possible?

marco

Re: Relocate KVstore files on filesystem

redflexigork — Wed, 13 Sep 2017 00:23:45 GMT

Added a stanza in etc\system\local\server.conf. It works, thanks.

Re: Relocate KVstore files on filesystem

nawazns5038 — Tue, 22 May 2018 18:03:03 GMT

How to implement this in a cluster ??

Re: Relocate KVstore files on filesystem

jimodonald — Tue, 22 May 2018 18:06:15 GMT

Put the server.conf into an app and deploy it from your cluster master just like any other app you’d deploy. However it’s generally not recommended to run a kvstore on your indexers.

Re: Relocate KVstore files on filesystem

mikelauth — Fri, 14 Jun 2019 17:55:06 GMT

This worked perfect! Thank you!