Knowledge Management

Relocate KVstore files on filesystem

jimodonald
Contributor

I am extensively using KV Store for a project and have run into an issue with where the files are stored on disk. There is a long sorted history to the disk layout, but I am not delving into that at this time.

The server is a Windows 2012 server running Splunk 6.3.3. The C: drive is 96GB in size (with 9.7GB free) and the F: drive is 2 TB in size (with 1.7 TB free). Splunk is installed to the C: drive as per corporate standards. My KV store is currently 60GB in size. My issue is that the KV store files exist in the default index location at $SPLUNK_HOME$\var\lib\splunk and I do not see any documentation related to relocating the KV store to a different filesystem like is possible with indexes.

I've tried to update the indexes.conf to point it to a different filesystem as follows:

[default]

[kvstore]
homePath   = F:\Splunk\var\lib\splunk\kvstore\db
coldPath   = F:\Splunk\var\lib\splunk\kvstore\colddb
thawedPath = $SPLUNK_DB\kvstore\thaweddb

However that does not move the mongo directory or the dumps directory under the kvstore directory.

I suspect (but have not tried yet) that updating the kvstore stanza to something like:

[kvstore]
mongoPath = F:\Splunk\var\lib\splunk\kvstore\mongo
dumpsPath = F:\Splunk\var\lib\splunk\kvstore\dumps

might to the trick, but thought I'd check with the larger audience before I go breaking stuff.

So how about it? Has anyone successfully relocated the kv store files by using indexes.conf? If so, how did you do it?

Thanks in advance.

1 Solution

jimodonald
Contributor

I have not tested this yet -- haven't had time yet, but I did find this stanza in server.conf:

[kvstore]
dbPath = <path>
 * Path where KV Store data is stored.
 * Changing this directory after initial startup does not move existing data.
  The contents of the directory should be manually moved to the new
  location.
 * Defaults to $SPLUNK_DB/kvstore.

so in theory you should be able to define the path for the kvstore in server.conf. If you try it out, please post back with your results.

UPDATE: This has been tested successfully and worked exactly like you'd expect it to. Just like moving an index.

View solution in original post

jimodonald
Contributor

Put the server.conf into an app and deploy it from your cluster master just like any other app you’d deploy. However it’s generally not recommended to run a kvstore on your indexers.

0 Karma

marcoscala
Builder

Good question! And what about relocating a single Collection inside the KVStore? I'm generating a sample collection for an app and I'd like to export just that collection instead of all the KV Store.
Is it possible?

marco

0 Karma

jimodonald
Contributor

I have not tested this yet -- haven't had time yet, but I did find this stanza in server.conf:

[kvstore]
dbPath = <path>
 * Path where KV Store data is stored.
 * Changing this directory after initial startup does not move existing data.
  The contents of the directory should be manually moved to the new
  location.
 * Defaults to $SPLUNK_DB/kvstore.

so in theory you should be able to define the path for the kvstore in server.conf. If you try it out, please post back with your results.

UPDATE: This has been tested successfully and worked exactly like you'd expect it to. Just like moving an index.

mikelauth
Explorer

This worked perfect! Thank you!

0 Karma

nawazns5038
Builder

How to implement this in a cluster ??

0 Karma

redflexigork
Explorer

Added a stanza in etc\system\local\server.conf. It works, thanks.

0 Karma

DavidHourani
Super Champion

hello did you find any solution for that ?

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...