SOLVED Importing existing backup in new app instance fails

[nebulon]

Moved from LDAP integration thread.

[imc67]

@nebulon alas after step 6 the GUI is a mess, it looks like CSS is not loading. I can login with LDAP but the GUI is one big mess without CSS.

What to do now?

[imc67]

@nebulon nowhere errors, but when I do login and try to find my way through the GUI mess (without CSS) I see this error in settings:

Rechten
Deze mappen moeten beschrijfbaar zijn door de webservergebruiker (www-data). Aanbevolen rechten: 775
storage/app/ 	OK
storage/framework/ 	OK
storage/framework/cache/data/ 	Non-writable files found
/app/code/storage/framework/cache/data/5e/3c/5e3cbbed47195384f39edf30b47d8cd8245cdff8

Run the following command (read more):
sudo chown -R www-data:www-data /app/code

I did the last sudo command but that didn't work out because its read only

[nebulon]

@imc67 that does not seem to have much to do with the LDAP integration though, but possibly something wrong with the backup importing. The storage folder should be symlinked into data. I've just retried it here and an import into a new instance worked again.

Did you use the import backup as outlined in https://docs.cloudron.io/backups/#import-app-backup ?

[imc67]

@nebulon said in Status LDAP integration?:

Did you use the import backup as outlined in https://docs.cloudron.io/backups/#import-app-backup ?

Yes, been there done that

What would happen if one change the sequence in steps, so (without LDAP license activation) create backup, import in new app, switch to correct domain, check if everything works and then activate license?

[nebulon]

That would work as well, however as said, the error does not seem to be related to LDAP module. Something didn't work out when installing the new instance or then importing the backup, since those folders should not be in read-only storage according to the package. Did you in any way interact with the freshly installed instance prior to importing the backup?

[imc67]

@nebulon said in Status LDAP integration?:

however as said, the error does not seem to be related to LDAP module

I understand, that's why I suggest to do a "normal" migration to a new instance to see if that works and then activate LDAP.

Did you in any way interact with the freshly installed instance prior to importing the backup?

No, didn't even logged in the first time, is that needed?

[imc67]

@nebulon just did every step you suggested again, this time no such 755 error but still same issue with no CSS and also no scripts (I guess because clearing cache within that scambled GUI didn't work).

[imc67]

@nebulon I tried 2 other attempts from start and none worked out. Last scenario I did login the new app before importing backup and then the GUI was "normal", right after import it was wrong again.

I'm lost....

The only thing I can think of is that I have the following modules active and maybe because of "phone home" to check licenses (they are only for 1 domain) it breaks the app? (Actually I doubt it, because the very first time I changed the subdomain to the current and that also didn't work).

Custom Folders
Followers
LDAP Integration
Spam Filter
Tags

[nebulon]

I probably would have to debug this on your system then, don't seem to be able to reproduce the issue here. If you use the webterminal into the app, can you see /app/code/storage being a symlink to /app/data/storage ?

I will move those topics to a new thread since this is more like a general restore issue it seems.

[imc67]

@nebulon very good you moved this to a new thread.

On another Cloudron I just migrated (only migrate with no single module) a Freescout instance (this one is rather new, only 4 weeks and only few emails) and that went perfect.

The Cloudron/Freescout app that has the issues is already from the very first version and updates went fine.
The app is 1.98GB in size according to Cloudron System Info, however, when I calculate size on SCP login, the /appsdata/[ID]/data/ folder is 2.11GB and the mysqldump is 189.8MB.

If you use the webterminal into the app, can you see /app/code/storage being a symlink to /app/data/storage ?

In the current app it is symlink (bright blue is symlink I guess) together with Modules.

I don't keep up the migrated Freescout up for longer than a few minutes because that one is fully functional and fetches email and response with autoanswers. Is there an easy way to stop cron to prevent mixing of data between 2 of the same instances?

[nebulon]

@imc67 right you would have to stop the app to avoid interfering with the mailboxes.

just out of curiosity, did you restart the failing app after import? I don't quite get why storage/framework/cache/ would not be writeable, if the symlinks are there.

You can also try to run sudo chown -R www-data:www-data /app/data since that would set the permissions for the files in the symlinked files. Although that would be done on an app restart anyways.

[imc67]

@nebulon

this time no such 755

After the second try (see above) there was no such error anymore, so I don’t think we should look into that direction. But also don’t know in what direction to think either

[imc67]

@nebulon hope you have time somewhere coming days to debug this issue together?

Summary:

1. Tried to follow you steps 5 times to migrate a Freescout instance (1.98GB + 189.8MB mysqldump, since the first version on Cloudron)
2. Only the 1st time not only failed the "CSS/scripts" but also somehow the symlink.
3. All the other 4 times only failed with a GUI without (looks like) CSS and some buttons (scripts?) are not working (login is possible with LDAP) see screenshot
4. A migration of Freescout on another Cloudron went fine (only 4 weeks old instance with hardly any mails).
5. Just as a debug scenario I just cloned the current app and that also didn't work, same result with GUI/CSS/script.

[imc67]

@nebulon hopefully you can find time this week to debug together as we now have the premium LDAP plug-in but can’t use it

[nebulon]

@imc67 actually I have tried again but was not able to reproduce your issue to be able to debug this. Maybe this is really somehow related to the backup size of your instance Not sure how to debug this properly.

[MooCloud_Matt]

@nebulon
we just migrate our main server to a new one, and we got the same issue.

i will message you the link to our instance so you can check that out.

[girish]

@moocloud_matt thanks for the access! I was able to figure out this mystery...

@nebulon We don't store symlinks in backups and it seems that there is a bunch of them in /app/data/public-modules. Not sure who creates them but a restore doesn't create those symlinks back. You couldn't reproduce it because you I am guessing you used tgz backup which keeps symlinks inside it but rsync backup will not store symlinks (there is no way to store symlinks in object storage without hacks). So, the package has to be fixed create those symlinks on startup as needed.

[girish]

I have fixed the code in 6.2 to preserve symlinks in rsync backups/restore

[imc67]

@girish said in Importing existing backup in new app instance fails:

I have fixed the code in 6.2 to preserve symlinks in rsync backups/restore

I can confirm that with 6.2.2 the backup and restore of FreeScout does work!

[MooCloud_Matt]

@imc67
yep, @girish fixed in seconds, like a superhero! was an issue with the system link in rsync, and I discovered the output of the error was on the app log file, not on the container that's why we couldn't find anything on the log in Cloudron initially.