Monitoring Splunk

Splunk backup and restore procedure?



I have attempted to backup and restore splunk by simply tar up the entire splunk directory and restore it to a fresh installation of splunk. However when I start splunk service, splunkd crashes after.

I will require to backup the following:
1) data and indexes
2) configuration settings, users 3) saved searches, reports, field extractions, tags..etc 4) splunk apps

My questions are:
1) Do I need to run the command to roll my hot buckets to warm buckets first?Currently in my "/opt/splunk/var/lib/splunk/defaultdb/db" directory I only see "hot_v1_X"

2) Can I backup/tar the splunk directory without stopping splunk services?

3) In particular,which are the files/directory that I should backup?

Tags (3)


You don't need to stop Splunk to backup the configs.

As far as the configs go, create a git repo in $SPLUNK_BACKUP (off box, hopefully) then add this cron job however often you want the configs backed up (hourly, daily, decade-ly)

rsync -vaz $SPLUNK_HOME/etc/ $SPLUNK_BACKUP/;
git commit -a -m "Configs as of $(date)";

Then you can revert back to whatever version of your configs you want by looking through git log for the date you want to revert to, running git checkout $COMMIT, copying it to your indexer(s), and bouncing splunkd


Any more details on this? I tried the following which did not work:

cd $SPLUNK_HOME/var/lib/splunk/

tar -zcvf mydatabase.tgz mydatabase

I then copied the .tgz to another splunk instance and untar'd it in the same location. Restarted Splunk and Splunk restarts with the Index disabled. Any ideas?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!