Splunk backup and restore procedure?

[remy06]

Hi,

I have attempted to backup and restore splunk by simply tar up the entire splunk directory and restore it to a fresh installation of splunk. However when I start splunk service, splunkd crashes after.

I will require to backup the following:
1) data and indexes
2) configuration settings, users 3) saved searches, reports, field extractions, tags..etc 4) splunk apps

My questions are:
1) Do I need to run the command to roll my hot buckets to warm buckets first?Currently in my "/opt/splunk/var/lib/splunk/defaultdb/db" directory I only see "hot_v1_X"

2) Can I backup/tar the splunk directory without stopping splunk services?

3) In particular,which are the files/directory that I should backup?

[edbolton]

You don't need to stop Splunk to backup the configs.

As far as the configs go, create a git repo in $SPLUNK_BACKUP (off box, hopefully) then add this cron job however often you want the configs backed up (hourly, daily, decade-ly)

rsync -vaz $SPLUNK_HOME/etc/ $SPLUNK_BACKUP/;
cd $SPLUNK_BACKUP;
git commit -a -m "Configs as of $(date)";

Then you can revert back to whatever version of your configs you want by looking through git log for the date you want to revert to, running git checkout $COMMIT, copying it to your indexer(s), and bouncing splunkd

[gkanapathy]

Please see the docs:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Backupindexeddata and http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Backupconfigurations

[TonyLeeVT]

Any more details on this? I tried the following which did not work:

cd $SPLUNK_HOME/var/lib/splunk/

tar -zcvf mydatabase.tgz mydatabase

I then copied the .tgz to another splunk instance and untar'd it in the same location. Restarted Splunk and Splunk restarts with the Index disabled. Any ideas?