Getting Data In

Is it appropriate to take VM snapshots prior to upgrading Splunk Deployment Server and Heavy Forwarder to 6.5.1?

cbaiocchetti
New Member

Hello all. Apologies in advance if the answer to these questions are documented elsewhere, but I've not been able to find any direct answers so far.

I am about to upgrade our Deployment Server and Heavy Forwarder to our cloud instance (each role on a separate Windows server VM) from 6.2.5 to 6.5.1. I've not upgraded these before, so am a little nervous. I'd like to take VM snapshots prior to upgrading and roll back if there are issues. Is it safe to do this? Is there a risk of data duplication from the Heavy Forwarder if a roll-back occurred? There are several hundred GB worth of Syslog files that we leave on that server and I'd hate for it all to show up twice is searches (not to mention blow our daily limit).

Also, on the Deployment Server, are there any specific directories that should be backed up/copied so that UF configurations can be preserved?

Thank you in advance for any help.

Regards,

Chris

0 Karma

coltwanger
Contributor

If by restoring the snapshot on the HF, you'd be restoring the syslog data stored on that server at the same time, I don't believe you'd risk duplication by restoring the snapshot. You would have lost the incoming data between the time the snapshot was taken and when it was restored (but potentially already indexed by Splunk during this time, so not really "lost").

If the syslog data won't be restored to the snapshot like Splunk will, then you risk duplication on ingested logs during the timespan between when the snapshot was taken, and when you decided to roll back. To reduce this risk, you can stop splunk before taking the snapshot, and you can stop splunk prior to the restoring the snapshot and backup the fishbucket folder. Once you backup the fishbucket, you should be able to restore the snapshot, and overwrite the old fishbucket with the new one. This should keep the pointers for the syslog data at what they were before restoring the snapshot.

You won't risk complete duplication of all data, just the data between the snapshot and restore if you don't back up the fishbucket prior to restoration.

For the Deployment Server, it's safest to zip the entire etc folder. But you're probably really only going to want the /etc/deployment-apps, /etc/apps, and /etc/system/ folders backed up.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...