Moving a single index (or multiple) away from the defaultdb

[beepboop12]

Currently in Splunk we have it set up in indexes.conf to explicitly request that "cartlog" (a specific index) go to its own db:

/opt/splunk/etc/apps/webintelligence/local/indexes.conf
[cartlog]
coldPath = $SPLUNK_DB/cartlog/colddb
homePath = $SPLUNK_DB/cartlog/db
thawedPath = $SPLUNK_DB/cartlog/thaweddb

However, we don't have it specified for quite a few things, and they are all going to defaultdb. My question is, is it possible to set up new dbs in indexes.conf and then MOVE only SOME of the indexes over to the new db?

For instance, say I have index1, index2, and index3 all being stored in defaultdb. What I really want is to create db1, db2, and db3, and then move their respective indexes over FROM defaultdb and continue to index to the correct directory/db.

So far I have read through http://answers.splunk.com/answers/65301/moving-the-index-database and the link provided therein (http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Configureindexstorage). Please let me know if this is possible or if more information for what I am trying to accomplish is needed.

[kristian_kolb · Accepted solution]

There might be some confusion of terminology. Basically an index is a 'db', where the defaultdb is the main index. $SPLUNK_DB usually refers to the 'root' directory for where the indexes are stored (default paths are /opt/splunk/var/lib/splunk/ on *nix and c:\program files\splunk\var\lib\splunk on Win).

You can set up new indexes, as it seems you have, either in the GUI or manually in indexes.conf. You can then route incoming event into this index, normally by setting the index=blahblah in inputs.conf under a monitor stanza.

Moving a whole index is not particularly hard. Simplest is to stop splunk, move the index directory to the new location, edit indexes.conf to reflect the new location (you can use absolute paths, not just relative to $SPLUNK_DB). Restart splunk.

However, it is quite hard to move data (individual events) from an index to another once it's there, but I don't think that was your question.

---

UPDATE:

Hmm, I'm still not sure what your situation is; you say that you have created some indexes (index1, index2 etc) in the defaultdb. I'm starting to think that maybe you correctly created indexes, but stored them in the directory that belongs to 'main', i.e. /opt/splunk/var/lib/splunk/defaultdb/yourindex.

If so, and if that is even possible without splunk complaining, it should be just as easy to move anyway. Just make sure that you don't move anything that belongs to defaultdb. All indexes should have the following subdirectories under it, colddb, db and thaweddb. Nothing more.

The actual events within an index are stored in $SPLUNK_DB/indexname/db or colddb/bucketfolder/rawdata/journal.gz The index (I just love the naming conventions) that make the event data searchable is the $SPLUNK_DB/indexname/db or colddb/bucketfolder/*.tsidx file (or files).
I have never tried to move anything out of this type of location in order to place it in another index.

Perhaps it would be a good thing if you clarified in which way your events are 'in the wrong db'.

/K

EDIT. Typos, clarification, etc.

[kristian_kolb]

see update above.
/K

[ChrisG]

Just as a follow-up to kristian.kolb's answer: For instructions about moving an index, see http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Moveanindex. If you haven't already read the topic about how the indexer stores data, that provides useful background: http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/HowSplunkstoresindexes.

[opticsplanet]

Hello kristian.kolb, thank you for the swift response. You mentioned that it is quite hard to move data from an index once it's there. How hard exactly? Have you ever done it or known someone to do it?