Deployment Architecture

What is the best practice to backup Splunk?

mlevsh
Builder

Hi,
We have Splunk v. 6.3.3 multi site indexer cluster (8 indexers), 4 search heads, deployment server.
What's the best approach to backup policy?
1. Should we backup $SPLUNK_HOME/etc directory on servers?
2. Will backing up of diag file be useful?
3. Any other advices?

Thank you in advance!

1 Solution

ehudb
Contributor

1.. Backup:

Search heads:

If the search heads are in a cluster,
- backup the deployer $SPLUNK_HOME\etc\shcluster
- $SPLUNK_HOME\etc of one the peers

Otherwise:
- backup $SPLUNK_HOME\etc of all the search heads

Indexers:
Backup warm\cold\frozen buckets
Also backup cluster master - master apps.

Deployment (Forwarder deployment?):
Backup $SPLUNK_HOME\etc directory

2..
I wouldn't backup diag file as it contains more than the needed files

3..
Docs about backup:
Indexed data (bcukets) - http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/Backupindexeddata
Configuration (etc) - http://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Backupconfigurations

View solution in original post

maraman_splunk
Splunk Employee
Splunk Employee

Hi,

 

for configuration, state and kvstore backup, you could use splunkconf-backup app (https://splunkbase.splunk.com/app/5600 , https://github.com/splunk/splunkconf-backup/wiki )

 

for indexed data,

  • in classic architecture, you mainly have to rely on traditional backup solutions (and their challenges)
  • in Smartstore architecture , you could leverage object store feature like versioning, lifecycle policy for delete, compliance option like objectlock and/or ability to sync object to another site in the background, depending on what you want to achieve

 

0 Karma

skoelpin
SplunkTrust
SplunkTrust

The main thing you want to backup is your hot/warm/cold/frozen db's which are under SPLUNK_HOME/var/lib/splunk/defaultdb/db and SPLUNK_HOME/var/lib/splunk/defaultdb/colddb along with your configs under SPLUNK_HOME/etc/system/local

Don't worry about backing up the diag's since it's just diagnostics on the current state of your system

How do you plan on backing this up? Are you running on a SAN? If so, do you plan on taking daily snapshots?

http://docs.splunk.com/Documentation/Splunk/6.2.0/Indexer/HowSplunkstoresindexes

ehudb
Contributor

1.. Backup:

Search heads:

If the search heads are in a cluster,
- backup the deployer $SPLUNK_HOME\etc\shcluster
- $SPLUNK_HOME\etc of one the peers

Otherwise:
- backup $SPLUNK_HOME\etc of all the search heads

Indexers:
Backup warm\cold\frozen buckets
Also backup cluster master - master apps.

Deployment (Forwarder deployment?):
Backup $SPLUNK_HOME\etc directory

2..
I wouldn't backup diag file as it contains more than the needed files

3..
Docs about backup:
Indexed data (bcukets) - http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/Backupindexeddata
Configuration (etc) - http://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Backupconfigurations

SamHTexas
Builder

Reg post below by ehudb. What else should I be backing up & how often? Is backing up the Index data really necessary? 

1.. Backup:

Search heads:

If the search heads are in a cluster,
- backup the deployer $SPLUNK_HOME\etc\shcluster
- $SPLUNK_HOME\etc of one the peers

Otherwise:
- backup $SPLUNK_HOME\etc of all the search heads

Indexers:
Backup warm\cold\frozen buckets
Also backup cluster master - master apps.

Deployment (Forwarder deployment?):
Backup $SPLUNK_HOME\etc directory

2..
I wouldn't backup diag file as it contains more than the needed files

3..
Docs about backup:
Indexed data (bcukets) - http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/Backupindexeddata
Configuration (etc) - http://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Backupconfigurations

View solution in original post

Tags (1)
0 Karma

mlevsh
Builder

@ehudb thank you!

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...