Unable to restore

[joseph]

@ivan-petro said in Unable to restore:

My Default Data Directory was migrated according to the Cloudron documentation (https://docs.cloudron.io/storage/#default-data-directory),

Did you move platformdata, appsdata and the boxdata (or just some of them?) .

Thing is HEtzner backup of full server will of course not include any of these if it's in some other disk .

Since the update to 8.3.0 was initiated previously, the data in platformdata atleast is already post 8.3.0 (or in some intermediate state) and the database versions of 8.2.4 won't be able to handle it. I am guess redis doesn't start because 8.3.0 has redis 7 and 8.2.4 had older redis.

Do you have Cloudron backups? Might be easiest to restore from Cloudron backups of 8.2.4 .

[ivan-petro]

@joseph ,

Did you move platformdata, appsdata and the boxdata (or just some of them?) .

yes, I moved everything according docs: https://docs.cloudron.io/storage/#default-data-directory
this is my setup for a long time already.

Do you have Cloudron backups? Might be easiest to restore from Cloudron backups of 8.2.4 .

do you mean restore of every apps? or whole cloudron? I can't see backup of whole Cloudron. only apps backups.

--
is there are way to ignore that apps can't start and make hard update to 8.3.0 to fix issues? or to 8.3.1?

[ivan-petro]

@joseph , I can't do anything now, and can't restores apps from backups.
all apps in state "Restarting - Waiting for platform to initialize"
due to redis issue.

How to force fix this? really need help 8 hours of outage

[joseph]

@ivan-petro from what i understand, the setup is not correct.

I am afraid to give suggestions which might result in loss of data. Can you give the contents of files below:

• /home/yellowtent/platformdata/VERSION
• /home/yellowtent/box/VERSION

Can you also tell me where you saved Cloudron Backups. This is the Backups view. Are they saved outside the server or were they in the default /var/backups ?

[ivan-petro]

@joseph , please check output:

root@tc-cloudron:~# cat /home/yellowtent/box/VERSION
8.2.4
root@tc-cloudron:~# cat /home/yellowtent/platformdata/VERSION
8.2.4root@tc-cloudron:~#

---

backups stored on AWS.
but I can't understand should I do manual restore of every app?) I have around 100 apps.

can I just make hard update of cloudron?

and I can't make restore of apps due to status "Restarting - Waiting for platform to initialize"
all apps in this state now.

[joseph]

@ivan-petro you said you moved the platformdata to somewhere else. the VERSION file is from that moved location, correct ?

So, if I understood correctly, you have reverted back both the main volume and the storage where you moved the (platformdata, appsdata, ...) storage .

If the above is correct, I am not even sure why it says "Waiting for platform to initialize" . Haven't you reverted everything to the old state? What am I missing? The confusion is that if everything is reverted to 8.2.4, that message should never appear.

[ivan-petro]

@joseph , my data is moved.
I did it several month ago according docs:
https://docs.cloudron.io/storage/#default-data-directory

it's symlinks. you may check:

root@tc-cloudron:/home/yellowtent# ls -lah
total 28K
drwxr-xr-x 4 yellowtent yellowtent 4.0K Feb 6 11:04 .
drwxr-xr-x 6 root root 4.0K Mar 9 2023 ..
lrwxrwxrwx 1 root root 34 Jan 23 00:58 appsdata -> /mnt/tc-cloudron-volume-1/appsdata
-rw-r--r-- 1 yellowtent yellowtent 220 Jan 6 2022 .bash_logout
-rw-r--r-- 1 yellowtent yellowtent 3.7K Jan 6 2022 .bashrc
drwxr-xr-x 8 yellowtent yellowtent 4.0K Feb 2 10:16 box
lrwxrwxrwx 1 root root 33 Jan 23 00:58 boxdata -> /mnt/tc-cloudron-volume-1/boxdata
-rw-r--r-- 1 yellowtent yellowtent 0 Dec 11 2022 .cloud-locale-test.skip
drwx------ 2 yellowtent yellowtent 4.0K Apr 2 2023 .gnupg
lrwxrwxrwx 1 root root 38 Jan 23 01:05 platformdata -> /mnt/tc-cloudron-volume-1/platformdata
-rw-r--r-- 1 yellowtent yellowtent 807 Jan 6 2022 .profile

[ivan-petro]

@joseph
"The confusion is that if everything is reverted to 8.2.4, that message should never appear"
yes it's interesting. can't understand wha's going on. I just can told you that I have setup like in your docs here:

https://docs.cloudron.io/storage/#default-data-directory

My platformdata , boxdata, appsdata on seperate volume.

I did restore from hetzner backup of main server disk.

[joseph]

@ivan-petro yes, so what I am wondering is if the separate volume was also restored to 8.2.4 state (i.e alongside the main server disk).

Think of it this way: you have mysql v6 (your main disk) . you store mysql's data in another volume. now you restored mysql with v4 (i.e restored your main disk). it won't start with the data in another volume. this is because the data is of a newer version i.e v6. this is what I am asking. this is possibly why redis is also not starting. most software won't work with old code and new data .

[ivan-petro]

@joseph , ok, how to resolve this?
is there are way to make hard update of cloudron? ignoring state of apps

[joseph]

@ivan-petro depending on the state maybe we have to hotfix the server with 8.3 . but this is mostly a developer tool. if you write to me on support@cloudron.io , i can try to hotfix the server.

[ivan-petro]

@joseph , Ok, I will write. just interesting if I can stop all apps by one command and make hard update to 8.3?

[joseph]

@ivan-petro it's also quite easy to just start afresh using the AWS backup . See https://docs.cloudron.io/backups/#restore-cloudron .

[ivan-petro]

@joseph , but I have separate volumes of apps(
please is it possible just update current to 8.3 ? I am wondering, I just need to ignore state of apps.

[ivan-petro]

@joseph ,

I've set up a new server and am in the process of restoring from AWS backups. However, I noticed that automatic backups were disabled for some apps, and the only available backups on AWS are quite old.

I have already switched Cloudron’s DNS to the new server. The old server is still active, and I would like to manually migrate specific apps from it to the new server.

Could you provide a step-by-step guide or tutorial on how to manually transfer these apps while preserving their data and configurations?

Thanks in advance for your help!

Best regards,
Ivan.

[joseph]

Oh, the whole Cloudron workflow/actions relies on backups. Not sure how much more I can help but let's see . You can try the below but we don't really have any guide or tutorial since what we have is based on backups.

• Copy the contents of /home/yellowtent/appsdata/<appid>/ from the old server into the new server. Copy this into some temp location . Say /tmp/app1/ or something . Do not copy directly into appsdata , this won't work. In that directory, you will see one config.json . If you see the "manifest" field, it will have the appstore id and version (not upstreamVersion).
• Install a fresh app in new server . You can put in the version field in the URL bar if it's not matching to what you noted above.
• After the app installed: Backups -> Import . Select Filesystem and put in /tmp/app1 . Then, select rsync as the format as well.

[ivan-petro]

@joseph , thanks for tutorial!
I did this , but in the end of import I have error:

External Error: Error loading fsmetadata.json:ENOENT: no such file or directory, open '/mnt/cloudron-volume-2/appsdata/46322ece-e828-4c6c-a4a2-e8391f2e395f/fsmetadata.json'

please let me know if this can be fixed

[joseph]

@ivan-petro ah yes, that file is usually created by rsync backup. can you create a dummy file in the root of the /tmp/app1/fsmetadata.json directory like this:

{
    "emptyDirs": [],
    "execFiles": [],
    "symlinks": []
}

Also, forgot to mention, maybe you can also chown -R yellowtent:yellowtent /tmp/app1 . You might get permission errors when importing otherwise.

[ivan-petro]

@joseph thanks for help! yes it helped. app imported , app in "running" state. but for some reason can't start app. it's n8n app.

error:

=> Setting permissions
Mar 21 10:03:56 => Starting N8N
Mar 21 10:03:57 Last session crashed
Mar 21 10:03:57 Permissions 0644 for n8n settings file /app/data/user/.n8n/config are too wide. This is ignored for now, but in the future n8n will attempt to change the permissions automatically. To automatically enforce correct permissions now set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true (recommended), or turn this check off set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=false.
Mar 21 10:03:57 User settings loaded from: /app/data/user/.n8n/config
Mar 21 10:04:00 => Healtheck error: Error: connect ECONNREFUSED 172.18.17.229:5678
Mar 21 10:04:07 Initializing n8n process
Mar 21 10:04:08 Migration "AddWebhookId1611144599516" failed, error: column "webhookId" of relation "webhook_entity" already exists
Mar 21 10:04:08 Migrations in progress, please do NOT stop the process.
Mar 21 10:04:08 Starting migration AddWebhookId1611144599516
Mar 21 10:04:08 There was an error running database migrations
Mar 21 10:04:08 column "webhookId" of relation "webhook_entity" already exists
Mar 21 10:04:08 n8n ready on 0.0.0.0, port 5678
Mar 21 10:04:10 => Healtheck error: Error: read ECONNRESET
Mar 21 10:04:10 => Ensure directories
Mar 21 10:04:10 => Loading configuration
Mar 21 10:04:10 => Setting permissions
Mar 21 10:04:10 => Starting N8N
Mar 21 10:04:11 Permissions 0644 for n8n settings file /app/data/user/.n8n/config are too wide. This is ignored for now, but in the future n8n will attempt to change the permissions automatically. To automatically enforce correct permissions now set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true (recommended), or turn this check off set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=false.
Mar 21 10:04:11 User settings loaded from: /app/data/user/.n8n/config
Mar 21 10:04:12 Last session crashed
Mar 21 10:04:12 BoxError: Unexpected response code or HTTP error when piping /home/yellowtent/appsdata/46322ece-e828-4c6c-a4a2-e8391f2e395f/postgresqldump to http://172.18.30.2:3000/databases/db46322ecee8284c6ca4a2e8391f2e395f/restore?access_token=b34f5810ef1bca9de0e9ae8f2cf579dc8c6dee98de19c0e678aab0133d9393324478b02763b955a2caf9eca347a812de339642143c9ddf5525e51081d31d190f7c7d170d3e5e88ec4a4a2986ad047d7dd93c121440852b7c63d5d3ba43f03304855d21ffe00ab24a4ad217c13e3d9181771d595d2d04e9556cc6a7c2ab3cecdb&username=user46322ecee8284c6ca4a2e8391f2e395f: status 500 complete false
Mar 21 10:04:20 => Healtheck error: Error: connect ECONNREFUSED 172.18.17.229:5678
Mar 21 10:04:22 Initializing n8n process
Mar 21 10:04:22 n8n ready on 0.0.0.0, port 5678
Mar 21 10:04:22 Migrations in progress, please do NOT stop the process.
Mar 21 10:04:22 Starting migration AddWebhookId1611144599516
Mar 21 10:04:22 Migration "AddWebhookId1611144599516" failed, error: column "webhookId" of relation "webhook_entity" already exists
Mar 21 10:04:22 There was an error running database migrations
Mar 21 10:04:22 column "webhookId" of relation "webhook_entity" already exists
Mar 21 10:04:24 => Ensure directories
Mar 21 10:04:24 => Loading configuration
Mar 21 10:04:24 => Setting permissions
Mar 21 10:04:25 => Starting N8N
Mar 21 10:04:25 Permissions 0644 for n8n settings file /app/data/user/.n8n/config are too wide. This is ignored for now, but in the future n8n will attempt to change the permissions automatically. To automatically enforce correct permissions now set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true (recommended), or turn this check off set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=false.
Mar 21 10:04:25 User settings loaded from: /app/data/user/.n8n/config
Mar 21 10:04:26 Last session crashed

[joseph]

@ivan-petro getting close! I can only guess that maybe the package version you have installed and the package version you had previously running were different. Can you check the step 2 in my previous steps (that you have the version correct) ?

Have you tried any other app btw?