Solved Manual Backup

[johan]

Following a problem with by my host provider, my Cloudron server was terminated and its volumes detached. I tried to create a new one with the volumes from the old one but some things were broken in the process. The result is that I've lost access to my Cloudron Dashboard and all efforts to restore it have been unsuccessful. I think it would be better to start from scratch than to try to repair this install.

Long story short, I want to start a new, clean Cloudron install on Ubuntu 20.04 LTS (I used previously Ubuntu 16 since it was an old install) and import the data from the previous server. Fortunately I have access to all my data in /home/yellowtent, I think only the Docker images were lost.

Basically I want to be able to move Cloudron to another server (as described here) but can't do it with Backup now from the Dashboard since I don't have access to it. I don't know what I have to move to keep everything intact.

How can manually backup and restore all of my data on a new Cloudron install ?
Can I do a complete backup of Cloudron using a script or a command line on the server? Will the "Cloudron Restore" work the same?

[girish]

@johan Interesting situation.. One thing is missing /var/lib/mysql, this is the MySQL database of the platform code. Do you have access to this?

/home/yellowtent has all the latest app data, so it should be possible to resurrect all the apps from that. Before I take you down that tricky restore process, do you happen to have access to any previous app backup (not sure where you setup the backup)? The default is in the file system in /var/backups

[johan]

@girish Thanks for this quick answer!

I do also have access to /var/lib/mysql which contains:

-rw-r----- 1 mysql mysql       56 Mar 10  2018 auto.cnf
drwxr-x--- 2 mysql mysql     4096 Sep 15 01:05 box
-rw------- 1 mysql mysql     1680 Dec  5 14:51 ca-key.pem
-rw-r--r-- 1 mysql mysql     1112 Dec  5 14:51 ca.pem
-rw-r--r-- 1 mysql mysql     1112 Dec  5 14:51 client-cert.pem
-rw------- 1 mysql mysql     1676 Dec  5 14:51 client-key.pem
-rw-r--r-- 1 mysql mysql        0 Dec  5 14:51 debian-5.7.flag
-rw-r----- 1 mysql mysql      898 Dec  5 14:56 ib_buffer_pool
-rw-r----- 1 mysql mysql 79691776 Dec  5 15:36 ibdata1
-rw-r----- 1 mysql mysql 50331648 Dec  5 15:36 ib_logfile0
-rw-r----- 1 mysql mysql 50331648 Dec  5 15:36 ib_logfile1
-rw-r----- 1 mysql mysql 12582912 Dec  5 15:33 ibtmp1
drwxr-x--- 2 mysql mysql     4096 Dec  5 14:51 mysql
-rw-r--r-- 1 mysql mysql        6 Dec  5 14:51 mysql_upgrade_info
drwxr-x--- 2 mysql mysql     4096 Dec  5 14:51 performance_schema
-rw------- 1 mysql mysql     1680 Dec  5 14:51 private_key.pem
-rw-r--r-- 1 mysql mysql      452 Dec  5 14:51 public_key.pem
-rw-r--r-- 1 mysql mysql     1112 Dec  5 14:51 server-cert.pem
-rw------- 1 mysql mysql     1676 Dec  5 14:51 server-key.pem
drwxr-x--- 2 mysql mysql    12288 Dec  5 14:51 sys

On the currently attached volume, the /var/backups directory seem to only contain old data:

-rw-r--r-- 1 root root    71680 Oct 19 06:25 alternatives.tar.0
-rw-r--r-- 1 root root     3576 Jul  8 06:25 alternatives.tar.1.gz
-rw-r--r-- 1 root root    31055 Sep 15 01:04 apt.extended_states.0
-rw-r--r-- 1 root root     3429 Jul 19 23:04 apt.extended_states.1.gz
-rw-r--r-- 1 root root     3522 Mar 23  2019 apt.extended_states.2.gz
-rw-r--r-- 1 root root      268 Jan 21  2019 dpkg.diversions.0
-rw-r--r-- 1 root root      140 Jan 21  2019 dpkg.diversions.1.gz
-rw-r--r-- 1 root root      140 Jan 21  2019 dpkg.diversions.2.gz
-rw-r--r-- 1 root root      140 Jan 21  2019 dpkg.diversions.3.gz
-rw-r--r-- 1 root root      140 Jan 21  2019 dpkg.diversions.4.gz
-rw-r--r-- 1 root root      140 Jan 21  2019 dpkg.diversions.5.gz
-rw-r--r-- 1 root root      140 Jan 21  2019 dpkg.diversions.6.gz
-rw-r--r-- 1 root root      135 Mar 10  2018 dpkg.statoverride.0
-rw-r--r-- 1 root root      129 Mar 10  2018 dpkg.statoverride.1.gz
-rw-r--r-- 1 root root      129 Mar 10  2018 dpkg.statoverride.2.gz
-rw-r--r-- 1 root root      129 Mar 10  2018 dpkg.statoverride.3.gz
-rw-r--r-- 1 root root      129 Mar 10  2018 dpkg.statoverride.4.gz
-rw-r--r-- 1 root root      129 Mar 10  2018 dpkg.statoverride.5.gz
-rw-r--r-- 1 root root      129 Mar 10  2018 dpkg.statoverride.6.gz
-rw-r--r-- 1 root root   683434 Oct 19 03:05 dpkg.status.0
-rw-r--r-- 1 root root   190301 Oct 19 03:05 dpkg.status.1.gz
-rw-r--r-- 1 root root   190301 Oct 19 03:05 dpkg.status.2.gz
-rw-r--r-- 1 root root   190301 Oct 19 03:05 dpkg.status.3.gz
-rw-r--r-- 1 root root   190301 Oct 19 03:05 dpkg.status.4.gz
-rw-r--r-- 1 root root   190301 Oct 19 03:05 dpkg.status.5.gz
-rw-r--r-- 1 root root   190301 Oct 19 03:05 dpkg.status.6.gz
-rw------- 1 root root      867 Jul 19 23:04 group.bak
-rw------- 1 root shadow    722 Jul 19 23:04 gshadow.bak
-rw------- 1 root root     1776 Jul 19 23:04 passwd.bak
-rw------- 1 root shadow   1046 Jul 19 23:04 shadow.bak

I think the actual backup directory was on a different volume, which I haven't managed to re-attach yet. (I can try again if that's the only way to restore "as is").

[girish]

@johan yes, if you can attach volume and get the backups, the restore is trivial.

[johan]

@girish I've managed to retrieve the data from the other volume, it contains the following directories:

drwx------   2 root root  4096 Feb 14  2019 builder
drwx------   4 root root  4096 Mar 10  2019 buildkit
drwx--x--x   3 root root  4096 Feb 14  2019 containerd
drwx------  33 root root  4096 Dec  2 04:04 containers
drwx--x--x  14 root root  4096 Feb  3  2019 docker
drwx------   3 root root  4096 Feb 14  2019 image
drwxr-x---   3 root root  4096 Feb 14  2019 network
drwx------ 389 root root 65536 Dec  2 04:04 overlay2
drwx------   4 root root  4096 Feb 14  2019 plugins
drwx------   2 root root  4096 Oct 19 03:05 runtimes
drwx------   2 root root  4096 Feb 14  2019 swarm
drwx------   2 root root  4096 Dec  2 04:03 tmp
drwx------   2 root root  4096 Feb 14  2019 trust
drwx------ 468 root root 53248 Dec  2 04:04 volumes

Does it match the expected content of the default /var/backups?

If that's the case, then what's the trivial path to restore from that?

[johan]

Also retrieved this:

drwxr-xr-x 32 yellowtent yellowtent 4096 Dec  4 05:30 .
drwxr-xr-x  5 root       root       4096 Aug 22 22:51 ..
drwxr-xr-x  2 yellowtent yellowtent 4096 Sep 29 03:00 2020-09-29-030002-435
drwxr-xr-x  2 yellowtent yellowtent 4096 Sep 29 03:00 2020-09-29-030002-449
drwxr-xr-x  2 yellowtent yellowtent 4096 Oct  9 03:00 2020-10-09-030002-673
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov  4 04:00 2020-11-04-040002-457
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov  4 04:00 2020-11-04-040002-469
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov  5 04:00 2020-11-05-040002-459
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov  7 04:00 2020-11-07-040002-400
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov  8 02:04 2020-11-08-020002-194
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 11 02:04 2020-11-11-020002-265
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 15 02:04 2020-11-15-020002-284
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 16 04:01 2020-11-16-040002-408
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 17 04:00 2020-11-17-040002-404
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 17 04:00 2020-11-17-040002-462
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 18 02:04 2020-11-18-020002-224
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 22 02:04 2020-11-22-020002-305
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 23 04:01 2020-11-23-040002-486
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 24 04:00 2020-11-24-040002-355
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 25 02:04 2020-11-25-020002-224
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 27 04:00 2020-11-27-040002-584
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 28 04:00 2020-11-28-040002-360
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 29 02:04 2020-11-29-020002-234
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 30 04:01 2020-11-30-040002-445
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 30 04:00 2020-11-30-040002-490
drwxr-xr-x  2 yellowtent yellowtent 4096 Nov 30 04:07 2020-11-30-040002-492
drwxr-xr-x  2 yellowtent yellowtent 4096 Dec  1 04:00 2020-12-01-040002-461
drwxr-xr-x  2 yellowtent yellowtent 4096 Dec  1 04:00 2020-12-01-040002-465
drwxr-xr-x  2 yellowtent yellowtent 4096 Dec  1 04:06 2020-12-01-040002-478
drwxr-xr-x  2 yellowtent yellowtent 4096 Dec  2 02:04 2020-12-02-020002-220
drwxr-xr-x  2 yellowtent yellowtent 4096 Dec  2 04:00 2020-12-02-040002-333
drwxr-xr-x  2 yellowtent yellowtent 4096 Dec  2 04:00 snapshot

It seems I now have all the data to do a proper restoration!

[girish]

@johan Indeed. So, all you need to do now is to copy the latest backup directory (2020-12-02-040002-333 in this case) into the new server to say /root/latestbackup/. Then install Cloudron in new server. There is a button at the bottom in the setupdns to restore. Click that and in the restore information, just select File System and provide the path of the backup on the server (like /root/latestbackup/).

See https://docs.cloudron.io/backups/#restore-cloudron for more information.

[girish]

@johan Also, I imagine the old cloudron was on 5.6.3 and not on Cloudron 6? You can make out by looking at the filename of the backup (it will be box_...v5.6.3.tar.gz). The new cloudron installation version has to match what was in old cloudron. So, when you install the new Cloudron, you have to install it as ./cloudron-setup --version 5.6.3. This is because it will otherwise install Cloudron 6 by default.

You can only upgrade to Ubuntu 20 after updating to Cloudron 6. Note that we have not published the procedure for upgrading Cloudron yet. Please hold off on upgrading to Ubuntu 20. Currently, only fresh installs on Ubuntu 20 will work (without following the procedure).

[johan]

Thank you so much for your help @girish! Indeed it now seems very easy to restore when doing a new install, which is currently my plan on a fresh VPS running Ubuntu 20.04 (GNU/Linux 5.4.0-1018-kvm x86_64). I'm temporarily stuck but that's a separate issue.

Yes backups are 5.6.3 so I'll use the ./cloudron-setup --version 5.6.3. Thanks! I was wondering about that but didn't find that in the documentation.

If I'm correct, once my new 5.6.3 install with data restore will be finished, I'll be back on track with the Dashboard and automatic upgrades.

[girish]

@johan said in Manual Backup:

If I'm correct, once my new 5.6.3 install with data restore will be finished, I'll be back on track with the Dashboard and automatic upgrades.

Yup, that's exactly right.

[girish]

@johan said in Manual Backup:

Thank you so much for your help @girish! Indeed it now seems very easy to restore when doing a new install, which is currently my plan on a fresh VPS running Ubuntu 20.04 (GNU/Linux 5.4.0-1018-kvm x86_64). I'm temporarily stuck but that's a separate issue.

You have to use Ubuntu 18 for Cloudron 5.6.3. Once you are back on track, you can upgrade to Ubuntu 20 after you update to Cloudron 6. Hope that makes sense!

[johan]

It worked! My instance and dashboard are back online with all my data restored. I can feel the awesomeness of Cloudron again.

In the end the install with ./cloudron-setup --version 5.6.3 seem to have worked on Ubuntu LTS (after a minor version downgrade of NGINX). Now that I've mastered the install/restore I can always go back to Ubuntu 18 if I have any problem.

Thank you again for your help.