Cloudron Forum

I'm trying to move Nextcloud to a new Cloudron server. On the original Cloudron server I made the data directory a symlink to another bigger disk:

root@my:/home/yellowtent/appsdata/dc3a15ad-9993-4e5c-9039-46e73d048341# ls -Alh data
lrwxrwxrwx 1 root root 19 Oct 28  2018 data -> /srv/nextcloud-data

This symlink is now part of the backup, which breaks the restore on the new server:

2022-09-28T19:14:39.287Z box:backupformat/rsync Recreating empty directories in {"localRoot":"/home/yellowtent/appsdata/dc3a15ad-9993-4e5c-9039-46e73d048341","layout":[]}
2022-09-28T19:14:39.290Z box:apptask run: app error for state pending_restore: BoxError: unable to symlink: EEXIST: file already exists, symlink '/srv/nextcloud-data' -> '/home/yellowtent/appsdata/dc3a15ad-9993-4e5c-9039-46e73d048341/data'
    at restoreFsMetadata (/home/yellowtent/box/src/backupformat/rsync.js:152:33)
    at async Object.download (/home/yellowtent/box/src/backupformat/rsync.js:232:5)
    at async download (/home/yellowtent/box/src/backuptask.js:75:5)
    at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:110:5)
    at async install (/home/yellowtent/box/src/apptask.js:398:9) {
  reason: 'FileSystem Error',
  details: {}
}

It looks like the data/ directory gets created first and then while restoring it already exists.

Any idea how I could restore this backup?

(And yes, I'm aware that this symlink method isn't officially supported - It was the only way to achieve having an external storage back in 2018).

@girish do you have any plans to provide 1.9 soon?

UMap would be fantastic to see!

Secure, end-to-end encrypted, and privacy respecting sync for your contacts, calendars, tasks and notes.

Easy to use and open source - it seamlessly integrates with your existing apps so you won't even notice you are using it!

https://www.etesync.com/

https://github.com/etesync/server

Interview

Thank @nebulon for the fast response and the update.

However, the update didn't went smooth. After the update all shares are lost and the settings button in Lychee leads into the error message "Server error or API not found". Looking at the browser console it tries to load /dist/user.css which is not available (404). Most probably this corresponds to the error found in the app log:

Jan 02 18:36:07 [Wed Jan 02 18:36:07.258759 2019] [php7:warn] [pid 14] [client 172.18.0.1:57854] PHP Warning: chmod(): Read-only file system in /app/code/php/database/update_030208.php on line 51, referer: XXX
Jan 02 18:36:07 [Wed Jan 02 18:36:07.258878 2019] [php7:warn] [pid 14] [client 172.18.0.1:57854] PHP Warning: touch(): Unable to create file /app/code/php/database/../../dist/user.css because Read-only file system in /app/code/php/database/update_030208.php on line 52, referer: XXX

And as well in the Lychee log (/plugins/Log/)

2019-01-02 18:36:07 - error - update_030108 (12) 	- Unknown column 'min_takestamp' in 'field list'
2019-01-02 18:36:07 - error - update_030108 (25) 	- Unknown column 'max_takestamp' in 'field list'
2019-01-02 18:36:07 - error - update_030203 (12) 	- Unknown column 'small' in 'field list'
2019-01-02 18:36:07 - error - update_030204 (12) 	- Unknown column 'lens' in 'field list'
2019-01-02 18:36:07 - error - update_030205 (12) 	- Unknown column 'license' in 'field list'
2019-01-02 18:36:07 - error - update_030207 (12) 	- Unknown column 'license' in 'field list'

Output of Lychee diagnostics:

Diagnostics
-----------
Warning: Dropbox import not working. No property for dropboxKey.
Error: 'uploads/small' is missing or has insufficient read/write privileges


System Information
------------------
/app/code/Lychee-front/package.json not found. Please do: git submodule init ; git submodule update
DB Version:             update_030208
System:                 Linux
PHP Version:            7.2
MySQL Version:          50723
Imagick:                1
Imagick Active:         1
Imagick Version:        1687
GD Version:             2.2.5
Plugins:                

As well /Lychee-front/images/password.svg is not found (404) because the Dockerfile downloads the packed Git repo and not the release Zip which contains all needed files.

Would you mind to have a look at this errors? Thank you very much...

Lychee is still at version 3.1.6 at the time of this writing. The current version is 3.2.8, see GitHub releases. Please note that Lychee has moved from electerious/Lychee to LycheeOrg/Lychee:

README:

Since the 1st of April 2018 this project has moved to it's own Organisation (https://github.com/LycheeOrg) where people are able to submit their fixes to it. We, the Organisation owners, want to thank electerious (Tobias Reich) for the opportunity to make this project live on.

I would love to see Lychee updated regularly, like the other apps on Cloudron...

A Prometheus-compatible ,or even better an OpenMetrics endpoint would be fantastic for a much improved monitoring system.

Regarding backup, please also have a look at https://forum.cloudron.io/topic/1575/backup-improvements-restic-as-backend. Thanks.

Yes, it worked. Unfortunately I did no document every step in detail, there were many steps involved. Here are they key steps, it's not a list which can be followed step-by-step, one has to figure out the details by himself:

• Export DB from old installation (mine was MariaDB)
• Start a temporary DB server on the Cloudron server and import the DB, something like:

docker run -d --name=mariadb -v mariadb-temp:/var/lib/mysql -v /tmp/dump:/data -p 3308:3306 -e MYSQL_ROOT_PASSWORD=password --network=cloudron mariadb:10.1

• Login to your Nextcloud container:

cloudron exec --app files.example.com -t bash

• Update the Nextcloud configuration to point to this temporary DB server and have the exact same version as the old installation. Example (replace values with the ones of your installation):

...
  'dbtype' => 'mysql',
  'version' => '13.0.0.14',
  'dbname' => 'nextcloud',
  'dbhost' => '172.18.0.10',
  'dbport' => '3306',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'root',
  'dbpassword' => 'password',
...

• Run migration:

sudo -u www-data php occ db:convert-type --clear-schema --all-apps --password ${POSTGRESQL_PASSWORD} pgsql ${POSTGRESQL_USERNAME} ${POSTGRESQL_HOST} ${POSTGRESQL_DATABASE}
sudo -u www-data php occ upgrade

• Move your data from the old to the new Cloudron server
• Remove temporary DB server and it's data

It's not very straight forward, but definitively doable.

Thanks @girish, that did the trick and lead straight into Backup Improvements: Restic as Backend.

My Cloudron server should store it's backup on a Raspberry Pi running offsite on a different place than the server. I've tried to do this with Minio running on the Raspberry Pi via a Wireguard VPN. While this basically works for a small amount of data, this doesn't work with a huge amount of data (currently ~1TB). The Minio server uses all available resources on the Raspberry Pi and Cloudron stops after 4h, stating the backup takes too long. I've also tried to increase this timeout by fiddling around in the code, but even after many hours the backup doesn't finish. The connection between the Cloudron server and the backup target has 1 Gbit/s, so bandwith is definitively enough.

I did some experiments with Restic and Minio, but the initial backup didn't finish after 8h waiting. So I decided to give Restic rest server a try, this worked much better and also caused much less load on the Raspberry Pi.

Feature request: Please integrate Restic as the backup tool into Cloudron. It has a huge user base and supports a lot of backends, no backend would have to be integrated manually into Cloudron again.

Thanks for considering this suggestion.

When moving data to an external location like described in the docs under Moving a single app's data directory to another location the symlinked location is not part of the backup. How can I change the backup configuration to also include this data?

total 242952
drwxr-xr-x 4 yellowtent yellowtent      4096 Oct 19 20:53 ./
drwxr-xr-x 6 root       root            4096 Oct 21 12:53 ../
-rw-r--r-- 1 yellowtent yellowtent      2814 Oct 23 10:01 config.json
lrwxrwxrwx 1 root       root              19 Oct 19 20:53 data -> /srv/nextcloud-data/
drwxrwx--- 9 www-data   www-data        4096 Oct 16 20:05 data_orig/
-rw-r--r-- 1 root       root             529 Oct 23 10:01 fsmetadata.json
-rw-r--r-- 1 yellowtent yellowtent 248753695 Oct 23 10:01 postgresqldump
drwxr-xr-x 2 lxd        lxd             4096 Oct 23 13:47 redis/
root@my:/home/yellowtent/appsdata/dc3a15ad-9993-4e5c-9039-46e73d048341# 

The folder data is missing in the backup, whereas data_orig is there.

I would love to see this app on Cloudron. As of this reply, the project seems to be in an intact and active state.

Would be great to see Fider on Cloudron.

Fider can help you collect and prioritize product feedback so that you can focus on building the right product.

@nebulon Thanks for the hints. I'll give it a try and will report back once done...

I want to migrate my Nextcloud installation to Cloudron but want to keep the already existing users (incl. password - I do not want to use the Cloudron authentication for Nextcloud) and settings (shares, permissions, extensions, ...). How can I do that?