Deployment Architecture
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

How to move internal indexes from the default location to a new directory in an indexer clustering environment?

Esky73
Builder

I have a distributed search environment consisting of 2 x indexers, 1 x master node, and a search head.

Currently, I am running out of disk space due to the _internal indexes which are on a smallish partition in the default location $SPLUNK_HOME/var/lib/splunk

I have a directory /app/splunk/data which has lots of free space and where all my app indexes are - I understand these are in this location because the indexes take precedence in the /etc/slave-apps/app/local/ directories.

I'm uncertain which index I need to change to move the _internal dbs from the default to the /app/splunk/data directory?

My understanding from the documentation is that best practice would be to make this change in the $SPLUNK_HOME/etc/master-apps/_cluster/local on the master node and push the config out?

0 Karma
1 Solution

jmallorquin
Builder

Hi,

Are you forwarding the internal events of the master to indexers? (recommended). When you did that, only have to have care about the location in the indexers.

You have to configure in the indexes.conf a new location for _audit _internal _introspection and the best way is using volume definition.

And correct, is in the master-apps/_cluster/local/indexes.conf

Hope i help you

View solution in original post

0 Karma

jmallorquin
Builder

Hi,

Are you forwarding the internal events of the master to indexers? (recommended). When you did that, only have to have care about the location in the indexers.

You have to configure in the indexes.conf a new location for _audit _internal _introspection and the best way is using volume definition.

And correct, is in the master-apps/_cluster/local/indexes.conf

Hope i help you

View solution in original post

0 Karma

Esky73
Builder

Doing a bit more research i found this article which says i am on the right track.

https://answers.splunk.com/answers/85446/how-to-move-internaldb-to-a-new-partition-in-a-clustered-in...

However - ive realised that if i make the change on the master node and its pushed out to all indexers - if the new location for the internal indexers is /app/splunk/data and that partition only exists on my 2 x indexers and not on the master node itself - will that cause an issue ?

Would adding indexes.conf to each indexer (only) pointing to the new location work ?

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!