Disaster recovery - Restoring a backup
I can't understand how I as an operator shall use the backup system Cloudron offers.
Yesterday I made a mistake and deleted an app I shouldn't have deleted (root cause).
Given that I only remember the name of the container, how can I restore the backup of the container?
I don't have the UUID, I don't have the BackupID. I have full access to the object storage, api keys, secrets and encryption key.
My object storage is teeming with backups, Cloudron is set to do it every 6 hours. And this succeeds day in and day out. Great!
However It is not clear to me how to restore these backups.
In my Object storage I have backup archives like this
They seem to follow the naming convention of the uuid for the contiainer, OK that makes sense.
When I am going to restore my backups I have to provide these fields:
As far as I can tell, as an operator I need to keep track of what was the UUID of the container I deleted, in order to restore it. In addition I must also know what is the BackupID. More on that.
When I upload one of my backup config files, there seems to be reference to a
For one of my backups, this is the BackupID
2020-06-13-064435-141/box_2020-06-13-065422-378_v5.2.4<--- If I don't have the Backup Config File how can I generate this ID? Or how to I find the ID of the container I accidentally deleted?
The only surefire way I can think of is to download the backup configuration file for every container and keep them in a folder on my onedrive for whenever disaster strikes such as now. Is this the way every one else are doing it?
Is there a place I can get an overview and piece together the variables and be able to do a restore of the container?
in order not to get stuck in the future, I now will need to do the following when I put an application in production (business critical):
- Create a spreadsheet containing a list of all my containers.
- Keep track of the
UUIDfor each container.
- Download the
backup configuration filefor each container.
- Explicitly store the
BackupIDin the spreadsheet per application.
- Maybe there are more fields I must keep track of as well (???).
- Randomly test the restore capability of each container monthly.
Unless there are some elegant solutions here?
Guys this was my first brush with disaster recovery for an application I host on Cloudron and it not go down well.
If it wasn't for my random DigitalOcean Droplet snapshot(backups) and DigitalOcean scheduled VM backups, I would have been screwed here. I was not able to find the right backup archive and the right values to enter to correctly restore backup.
If anything this is a defect toward the documentation, because I cannot figure it out by reading this: https://cloudron.io/documentation/backups/ document. All of the great backup functionality in Cloudron didn't help me.
In Risk Management we use the Bowtie risk causal model and focus on our Barrier Management. The backups of Cloudron was supposed to be a recovery barrier.
This is not uncommon in IT, disaster recovery is a key think to test out. Unfortunately for me I hadn't done all the testing necessary as I didn't see myself ending up in this scenario (making a mistake and deleting a container I actually needed).
As far as I can tell, as an operator I need to keep track of what was the UUID of the container I deleted, in order to restore it.
Sorry for much confusion. Let me first try to explain how to import.
Finding the App's ID
Each Cloudron app has a unique ID. This is separate from the internal docker container id and you should never have to use any docker container id related stuff,
Now the question is how to figure out this unique ID of an uninstalled app. One way is to look into the event log. Select the uninstalled apps as the filter:
Then when you click on the event, you will see the app id:
Locating the backup
The backups are stored with the filename
timestamp/app_<id>_v<packageversion>.tar.gz(for tgz backups) or
timestamp/app_<id>_v<packageversion>(for rsync backups). The package version is needed in the case where you have a old version of an app and you want to restore to that old version.
Install an empty app
We have to next install the app for appstore. For this go to the appstore and install the app.
Notice the package version in the URL bar. Above, I am trying to install package v2.0.5 of wordpress. This version must match the version in the backup we located above.
Finally, go to app -> configure -> import
OK, all that said, what can be improved here. I see that the way to locate an uninstalled app's id is missing from the docs. I can add that part. Is there anything else we can fix on our side? Maybe we can keep the app's fqdn -> uuid listing alongside the backups?