Data backup

[tekebede]

Hello,

what is the recommended way to automate a db backup to QNAP NAS every 24 hrs. (SPLUNK enterprise 8.0.4). is there a way to do it from the GUI or it must be done at the CLI. The server is running low on disk space and i have a 24 TB qnap sitting around.

[rabbidroid]

You can only back up warm or cold buckets. Here is a quick and dirty way to do it:

If you are running Linux, this will copy all non-hot buckets created in the last day to /tmp. replace /tmp with you desired target directory

warm_buckets=$(find /opt/splunk/var/lib/splunk -mmin -1440 -type d -name "db_*")

for i in $warm_buckets; do mkdir -p /tmp/$i/rawdata; done

for i in $warm_buckets; do rsync -auvn "$i"/*/journal.gz /tmp$i; done

Hope this helps

[tekebede]

I have successfully completed configuring the scsi  connection and  it shows as a disk on the server where spunk resides on. i managed to move about 200 GB of files from the warm bucket temporarily (manually) but i want to automate the process.  I will do that on the cold bucket once i figure out how to increase the rolling time from  the worm bucket to the cold and also roll out time from the hot to worm. 

Grateful for your help

Tess

[MaverickT]

There is no way of doing Splunk database backup via GUI. Even the CLI might get a little bit complicated, because you have to roll the buckets before making backup. You can learn more about making backup here:

https://docs.splunk.com/Documentation/Splunk/8.0.5/Indexer/Backupindexeddata

https://docs.splunk.com/Documentation/Splunk/8.0.5/Admin/BackupKVstore

Maybe you should consider moving the cold buckets to the  QNAP NAS?

[tekebede]

Thanks, MaverickT

I was thinking if I reduce the retention policy on the tsidx files the aged files will roll to cold bucket and i can move that to the NAS. but how do I move the cold buckets to the NAS server? both servers are on the same vlan.

[isoutamo]

It depends what OS you are using for splunk (Linux, Windows) and is it distributed or only all in one solution?

Anyhow 1st thing is to ensure that QNAS has enough IOPS for splunk (800 is minimum). If not then you should forget it. If it has and you can trust that it servers your otherwise enough well then you could go forward. 

Probably best (or not worst) option is define iSCSI LUNs on QNAP and then present those to your splunk indexers (use at least two network path). Add those to own "volume group" and add splunk volume over it. Then just configure splunk's indexes to use it. Then use that volume as cold storege for your data.

r. Ismo

[tekebede]

Mr. Soutamo

Sorry  for the delay but i am waiting for the network guys to drop a cable so i can make a SCASI connection and configure the QNAP. but what i didn't understand is when you say " present those to your splunk indexers use at least two network path. (Add those to own "volume group" and add splunk volume over it). Then just configure splunk's indexes to use it. how do i present that to SPLUNK? as you can see i am a rookie on both splunk and qnap