Restoring backups not working - stuck on Restoring

[jdaviescoates]

Last night I updated my Minio to the latest version v2.0.0-1. It was all working fine, but then I somehow messed it up, so now I'm trying to restore to the previous version v1.165.0 so I can start again, but it's not working

It's just stuck on Restoring and nothing is happening

Here's what it says in the logs:

May 11 22:49:52 box:tasks update 8145: {"percent":65,"message":"Downloading backup 2022-05-11-000134-100/app_wordpress-backups.ud.coop_v1.165.0"}
May 11 22:49:52 box:storage/filesystem download: /mnt/backup/2022-05-11-000134-100/app_wordpress-backups.ud.coop_v1.165.0.tar.gz
May 11 22:50:02 box:tasks update 8145: {"percent":65,"message":"Downloading 867M@83MBps"}
May 11 22:50:12 box:tasks update 8145: {"percent":65,"message":"Downloading 1737M@85MBps"}
May 11 22:50:22 box:tasks update 8145: {"percent":65,"message":"Downloading 2566M@84MBps"}
May 11 22:50:32 box:tasks update 8145: {"percent":65,"message":"Downloading 3420M@84MBps"}
May 11 22:50:42 box:tasks update 8145: {"percent":65,"message":"Downloading 4243M@84MBps"}
May 11 22:50:52 box:tasks update 8145: {"percent":65,"message":"Downloading 5065M@84MBps"}
May 11 22:51:02 box:tasks update 8145: {"percent":65,"message":"Downloading 5942M@84MBps"}
May 11 22:51:12 box:tasks update 8145: {"percent":65,"message":"Downloading 6829M@85MBps"}
May 11 22:51:22 box:tasks update 8145: {"percent":65,"message":"Downloading 7779M@86MBps"}
May 11 22:51:32 box:tasks update 8145: {"percent":65,"message":"Downloading 8676M@86MBps"}
May 11 22:51:42 box:tasks update 8145: {"percent":65,"message":"Downloading 9576M@87MBps"}
May 11 22:51:52 box:tasks update 8145: {"percent":65,"message":"Downloading 10459M@87MBps"}
May 11 22:51:55 box:tasks update 8145: {"percent":65,"message":"Downloading 10750M@87MBps"}
May 11 22:51:55 box:backuptask tarExtract: done.
May 11 22:51:55 box:backuptask downloadApp: time: 123.345
May 11 22:51:55 box:tasks update 8145: {"percent":70,"message":"Restoring addons"}
May 11 22:51:55 box:services restoreAddons
May 11 22:51:55 box:services restoreAddons: restoring ["localstorage"]
May 11 22:51:55 box:tasks update 8145: {"percent":80,"message":"Creating container"}
May 11 22:51:55 box:apptask createContainer: creating container
May 11 22:51:55 box:shell addLogrotateConfig spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh add 62102342-80df-4ed9-9c39-b5d86b80ec13 /tmp/62102342-80df-4ed9-9c39-b5d86b80ec13.logrotate
May 11 22:51:55 box:shell addCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh add 62102342-80df-4ed9-9c39-b5d86b80ec13
May 11 22:51:55 box:shell addCollectdProfile (stdout): Restarting collectd
May 11 22:51:55 box:apptask startApp: starting container
May 11 22:51:56 box:tasks update 8145: {"percent":95,"message":"Configuring reverse proxy"}
May 11 22:51:56 ==> Changing ownership
May 11 22:51:56 ==> Starting minio
May 11 22:51:56 box:reverseproxy ensureCertificate: wordpress-backups.ud.coop certificate already exists at /home/yellowtent/platformdata/nginx/cert/_.ud.coop.key
May 11 22:51:56 box:reverseproxy expiryDate: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert notAfter=Jul 24 23:01:53 2022 GMT daysLeft=74.04856583333333
May 11 22:51:56 box:reverseproxy providerMatchesSync: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert subject=CN = *.ud.coop domain=*.ud.coop issuer=C = US, O = Let's Encrypt, CN = R3 wildcard=true/true prod=true/true issuerMismatch=false wildcardMismatch=false match=true
May 11 22:51:56 box:reverseproxy ensureCertificate: minio-api.ud.coop certificate already exists at /home/yellowtent/platformdata/nginx/cert/_.ud.coop.key
May 11 22:51:56 box:reverseproxy expiryDate: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert notAfter=Jul 24 23:01:53 2022 GMT daysLeft=74.04856550925926
May 11 22:51:56 box:reverseproxy providerMatchesSync: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert subject=CN = *.ud.coop domain=*.ud.coop issuer=C = US, O = Let's Encrypt, CN = R3 wildcard=true/true prod=true/true issuerMismatch=false wildcardMismatch=false match=true
May 11 22:51:56 box:reverseproxy writeAppNginxConfig: writing config for "wordpress-backups.ud.coop" to /home/yellowtent/platformdata/nginx/applications/62102342-80df-4ed9-9c39-b5d86b80ec13.conf with options {"sourceDir":"/home/yellowtent/box","vhost":"wordpress-backups.ud.coop","hasIPv6":true,"ip":"172.18.19.93","port":8000,"endpoint":"app","redirectTo":null,"certFilePath":"/home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert","keyFilePath":"/home/yellowtent/platformdata/nginx/cert/_.ud.coop.key","robotsTxtQuoted":null,"cspQuoted":null,"hideHeaders":[],"proxyAuth":{"enabled":false,"id":"62102342-80df-4ed9-9c39-b5d86b80ec13","location":"/"},"ocsp":true}
May 11 22:51:56 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 11 22:51:57 box:apptask run: app error for state pending_restore: TypeError: Cannot read properties of undefined (reading 'API_SERVER_DOMAIN')
at writeAppNginxConfig (/home/yellowtent/box/src/reverseproxy.js:528:47)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async writeAppConfig (/home/yellowtent/box/src/reverseproxy.js:557:9)
at async Object.configureApp (/home/yellowtent/box/src/reverseproxy.js:574:5)
at async install (/home/yellowtent/box/src/apptask.js:407:5)
May 11 22:51:57 box:taskworker Task took 131.924 seconds
May 11 22:51:57 box:tasks setCompleted - 8145: {"result":null,"error":{"stack":"TypeError: Cannot set properties of undefined (setting 'taskId')\n at makeTaskError (/home/yellowtent/box/src/apptask.js:54:26)\n at run (/home/yellowtent/box/src/apptask.js:814:86)\n at runMicrotasks (<anonymous>)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)","message":"Cannot set properties of undefined (setting 'taskId')"}}
May 11 22:51:57 box:tasks update 8145: {"percent":100,"result":null,"error":{"stack":"TypeError: Cannot set properties of undefined (setting 'taskId')\n at makeTaskError (/home/yellowtent/box/src/apptask.js:54:26)\n at run (/home/yellowtent/box/src/apptask.js:814:86)\n at runMicrotasks (<anonymous>)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)","message":"Cannot set properties of undefined (setting 'taskId')"}}
Cannot set properties of undefined (setting 'taskId')
at makeTaskError (/home/yellowtent/box/src/apptask.js:54:26)
at run (/home/yellowtent/box/src/apptask.js:814:86)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
May 11 22:51:57 IAM initialization complete

It looks like it's trying to do the API domain thing, even though there is no such thing in this older version:

May 11 22:51:57 box:apptask run: app error for state pending_restore: TypeError: Cannot read properties of undefined (reading 'API_SERVER_DOMAIN')

[jdaviescoates]

Maybe I'm just not waiting long enough for the restore to finish?

I got tired of waiting and so went and tried to restore the next oldest backup:

May 11 23:04:19 box:settings initCache: pre-load settings
May 11 23:04:19 box:taskworker Starting task 8146. Logs are at /home/yellowtent/platformdata/logs/62102342-80df-4ed9-9c39-b5d86b80ec13/apptask.log
May 11 23:04:19 box:apptask run: startTask installationState: pending_restore runState: running
May 11 23:04:19 box:tasks update 8146: {"percent":10,"message":"Cleaning up old install"}
May 11 23:04:19 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 11 23:04:20 box:apptask deleteContainer: deleting app containers (app, scheduler)
May 11 23:04:20 box:shell removeCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh remove 62102342-80df-4ed9-9c39-b5d86b80ec13
May 11 23:04:20 box:shell removeCollectdProfile (stdout): Restarting collectd
May 11 23:04:20 box:shell removeCollectdProfile (stdout): Removing collectd stats of 62102342-80df-4ed9-9c39-b5d86b80ec13
May 11 23:04:20 box:shell removeLogrotateConfig spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh remove 62102342-80df-4ed9-9c39-b5d86b80ec13
May 11 23:04:20 Exiting on signal: TERMINATED
May 11 23:04:21
May 11 23:04:21 box:services teardownAddons: Tearing down []
May 11 23:04:21 box:tasks update 8146: {"percent":20,"message":"Downloading icon"}
May 11 23:04:21 box:apptask downloadIcon: Downloading icon of io.minio.cloudronapp@1.165.0
May 11 23:04:22 box:tasks update 8146: {"percent":40,"message":"Downloading image"}
May 11 23:04:22 box:docker downloadImage cloudron/io.minio.cloudronapp:20210623-170632-719620233
May 11 23:04:22 box:docker downloadImage: pulling image. attempt 1
May 11 23:04:22 box:docker pullImage: will pull cloudron/io.minio.cloudronapp:20210623-170632-719620233. auth: no
May 11 23:04:24 box:docker pullImage: {"status":"Pulling from cloudron/io.minio.cloudronapp","id":"20210623-170632-719620233"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"83ee3a23efb7"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"db98fc6f11f0"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"f611acd52c6c"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"dcded93e417d"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"87b63eef5647"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"006937d76cc6"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"8dab9e9d2594"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"2d86b342da5c"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"3bced370dd4c"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"038d1280ed6d"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"71963bcc2a55"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"62a3a16fcb86"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"f6927aea9342"}
May 11 23:04:24 box:docker pullImage: {"status":"Already exists","progressDetail":{},"id":"88a76cf7025c"}
May 11 23:04:24 box:docker pullImage: {"status":"Pulling fs layer","progressDetail":{},"id":"da2c5c854c67"}
May 11 23:04:24 box:docker pullImage: {"status":"Pulling fs layer","progressDetail":{},"id":"a16631bc6f4e"}
May 11 23:04:24 box:docker pullImage: {"status":"Pulling fs layer","progressDetail":{},"id":"a124cda54c5a"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":664,"total":664},"progress":"[==================================================>] 664B/664B","id":"a16631bc6f4e"}
May 11 23:04:25 box:docker pullImage: {"status":"Download complete","progressDetail":{},"id":"a16631bc6f4e"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":671,"total":774},"progress":"[===========================================> ] 671B/774B","id":"a124cda54c5a"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":774,"total":774},"progress":"[==================================================>] 774B/774B","id":"a124cda54c5a"}
May 11 23:04:25 box:docker pullImage: {"status":"Verifying Checksum","progressDetail":{},"id":"a124cda54c5a"}
May 11 23:04:25 box:docker pullImage: {"status":"Download complete","progressDetail":{},"id":"a124cda54c5a"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":187740,"total":18430047},"progress":"[> ] 187.7kB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":5353971,"total":18430047},"progress":"[==============> ] 5.354MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":9933299,"total":18430047},"progress":"[==========================> ] 9.933MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":14152179,"total":18430047},"progress":"[======================================> ] 14.15MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Downloading","progressDetail":{"current":16441843,"total":18430047},"progress":"[============================================> ] 16.44MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Verifying Checksum","progressDetail":{},"id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Download complete","progressDetail":{},"id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":196608,"total":18430047},"progress":"[> ] 196.6kB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":1572864,"total":18430047},"progress":"[====> ] 1.573MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:25 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":2752512,"total":18430047},"progress":"[=======> ] 2.753MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":4325376,"total":18430047},"progress":"[===========> ] 4.325MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":5701632,"total":18430047},"progress":"[===============> ] 5.702MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":7274496,"total":18430047},"progress":"[===================> ] 7.274MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":8847360,"total":18430047},"progress":"[========================> ] 8.847MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":10223616,"total":18430047},"progress":"[===========================> ] 10.22MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":11403264,"total":18430047},"progress":"[==============================> ] 11.4MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":12976128,"total":18430047},"progress":"[===================================> ] 12.98MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":13959168,"total":18430047},"progress":"[=====================================> ] 13.96MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:26 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":15925248,"total":18430047},"progress":"[===========================================> ] 15.93MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:27 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":17498112,"total":18430047},"progress":"[===============================================> ] 17.5MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:27 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":18430047,"total":18430047},"progress":"[==================================================>] 18.43MB/18.43MB","id":"da2c5c854c67"}
May 11 23:04:27 box:docker pullImage: {"status":"Pull complete","progressDetail":{},"id":"da2c5c854c67"}
May 11 23:04:27 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":664,"total":664},"progress":"[==================================================>] 664B/664B","id":"a16631bc6f4e"}
May 11 23:04:27 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":664,"total":664},"progress":"[==================================================>] 664B/664B","id":"a16631bc6f4e"}
May 11 23:04:27 box:docker pullImage: {"status":"Pull complete","progressDetail":{},"id":"a16631bc6f4e"}
May 11 23:04:27 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":774,"total":774},"progress":"[==================================================>] 774B/774B","id":"a124cda54c5a"}
May 11 23:04:27 box:docker pullImage: {"status":"Extracting","progressDetail":{"current":774,"total":774},"progress":"[==================================================>] 774B/774B","id":"a124cda54c5a"}
May 11 23:04:27 box:docker pullImage: {"status":"Pull complete","progressDetail":{},"id":"a124cda54c5a"}
May 11 23:04:27 box:docker pullImage: {"status":"Digest: sha256:6ea9756d8ad0e658e381b1ecbe06e1c5210af35a70893b71db2a1c4a52cfeb68"}
May 11 23:04:27 box:docker pullImage: {"status":"Status: Downloaded newer image for cloudron/io.minio.cloudronapp:20210623-170632-719620233"}
May 11 23:04:27 box:docker downloaded image cloudron/io.minio.cloudronapp:20210623-170632-719620233
May 11 23:04:27 box:tasks update 8146: {"percent":50,"message":"Creating app data directory"}
May 11 23:04:27 box:tasks update 8146: {"percent":65,"message":"Downloading backup and restoring addons"}
May 11 23:04:27 box:services setupAddons: Setting up ["localstorage"]
May 11 23:04:27 box:services setupAddons: setting up addon localstorage with options {}
May 11 23:04:27 box:services setupLocalStorage
May 11 23:04:27 box:shell createVolume spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/mkdirvolume.sh /home/yellowtent/appsdata/62102342-80df-4ed9-9c39-b5d86b80ec13/data
May 11 23:04:27 box:services clearAddons
May 11 23:04:27 box:services clearAddons: clearing ["localstorage"]
May 11 23:04:27 box:services clearLocalStorage
May 11 23:04:27 box:shell clearVolume spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/clearvolume.sh clear /home/yellowtent/appsdata/62102342-80df-4ed9-9c39-b5d86b80ec13/data
May 11 23:04:27 box:backuptask download: Downloading 2022-05-10-010001-091/app_wordpress-backups.ud.coop_v1.165.0 of format tgz to {"localRoot":"/home/yellowtent/appsdata/62102342-80df-4ed9-9c39-b5d86b80ec13","layout":[]}
May 11 23:04:27 box:tasks update 8146: {"percent":65,"message":"Downloading backup 2022-05-10-010001-091/app_wordpress-backups.ud.coop_v1.165.0"}
May 11 23:04:27 box:storage/filesystem download: /mnt/backup/2022-05-10-010001-091/app_wordpress-backups.ud.coop_v1.165.0.tar.gz
May 11 23:04:37 box:tasks update 8146: {"percent":65,"message":"Downloading 799M@76MBps"}
May 11 23:04:47 box:tasks update 8146: {"percent":65,"message":"Downloading 1604M@78MBps"}
May 11 23:04:57 box:tasks update 8146: {"percent":65,"message":"Downloading 2498M@82MBps"}
May 11 23:05:07 box:tasks update 8146: {"percent":65,"message":"Downloading 3289M@81MBps"}
May 11 23:05:17 box:tasks update 8146: {"percent":65,"message":"Downloading 4236M@84MBps"}
May 11 23:05:27 box:tasks update 8146: {"percent":65,"message":"Downloading 5131M@85MBps"}
May 11 23:05:37 box:tasks update 8146: {"percent":65,"message":"Downloading 6068M@86MBps"}
May 11 23:05:47 box:tasks update 8146: {"percent":65,"message":"Downloading 6908M@86MBps"}
May 11 23:05:57 box:tasks update 8146: {"percent":65,"message":"Downloading 7721M@85MBps"}
May 11 23:06:07 box:tasks update 8146: {"percent":65,"message":"Downloading 8552M@85MBps"}
May 11 23:06:17 box:tasks update 8146: {"percent":65,"message":"Downloading 9383M@85MBps"}
May 11 23:06:27 box:tasks update 8146: {"percent":65,"message":"Downloading 10185M@85MBps"}
May 11 23:06:34 box:tasks update 8146: {"percent":65,"message":"Downloading 10750M@84MBps"}
May 11 23:06:34 box:backuptask tarExtract: done.
May 11 23:06:34 box:backuptask downloadApp: time: 126.857
May 11 23:06:34 box:tasks update 8146: {"percent":70,"message":"Restoring addons"}
May 11 23:06:34 box:services restoreAddons
May 11 23:06:34 box:services restoreAddons: restoring ["localstorage"]
May 11 23:06:34 box:tasks update 8146: {"percent":80,"message":"Creating container"}
May 11 23:06:34 box:apptask createContainer: creating container
May 11 23:06:34 box:shell addLogrotateConfig spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh add 62102342-80df-4ed9-9c39-b5d86b80ec13 /tmp/62102342-80df-4ed9-9c39-b5d86b80ec13.logrotate
May 11 23:06:34 box:shell addCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh add 62102342-80df-4ed9-9c39-b5d86b80ec13
May 11 23:06:34 box:shell addCollectdProfile (stdout): Restarting collectd
May 11 23:06:34 box:apptask startApp: starting container
May 11 23:06:35 box:tasks update 8146: {"percent":95,"message":"Configuring reverse proxy"}
May 11 23:06:35 ==> Changing ownership
May 11 23:06:35 ==> Starting minio
May 11 23:06:35 box:reverseproxy ensureCertificate: wordpress-backups.ud.coop certificate already exists at /home/yellowtent/platformdata/nginx/cert/_.ud.coop.key
May 11 23:06:35 box:reverseproxy expiryDate: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert notAfter=Jul 24 23:01:53 2022 GMT daysLeft=74.038395
May 11 23:06:35 box:reverseproxy providerMatchesSync: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert subject=CN = *.ud.coop domain=*.ud.coop issuer=C = US, O = Let's Encrypt, CN = R3 wildcard=true/true prod=true/true issuerMismatch=false wildcardMismatch=false match=true
May 11 23:06:35 box:reverseproxy ensureCertificate: minio-api.ud.coop certificate already exists at /home/yellowtent/platformdata/nginx/cert/_.ud.coop.key
May 11 23:06:35 box:reverseproxy expiryDate: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert notAfter=Jul 24 23:01:53 2022 GMT daysLeft=74.03839462962964
May 11 23:06:35 box:reverseproxy providerMatchesSync: /home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert subject=CN = *.ud.coop domain=*.ud.coop issuer=C = US, O = Let's Encrypt, CN = R3 wildcard=true/true prod=true/true issuerMismatch=false wildcardMismatch=false match=true
May 11 23:06:35 box:reverseproxy writeAppNginxConfig: writing config for "wordpress-backups.ud.coop" to /home/yellowtent/platformdata/nginx/applications/62102342-80df-4ed9-9c39-b5d86b80ec13.conf with options {"sourceDir":"/home/yellowtent/box","vhost":"wordpress-backups.ud.coop","hasIPv6":true,"ip":"172.18.18.156","port":8000,"endpoint":"app","redirectTo":null,"certFilePath":"/home/yellowtent/platformdata/nginx/cert/_.ud.coop.cert","keyFilePath":"/home/yellowtent/platformdata/nginx/cert/_.ud.coop.key","robotsTxtQuoted":null,"cspQuoted":null,"hideHeaders":[],"proxyAuth":{"enabled":false,"id":"62102342-80df-4ed9-9c39-b5d86b80ec13","location":"/"},"ocsp":true}
May 11 23:06:35 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 11 23:06:35 box:apptask run: app error for state pending_restore: TypeError: Cannot read properties of undefined (reading 'API_SERVER_DOMAIN')
at writeAppNginxConfig (/home/yellowtent/box/src/reverseproxy.js:528:47)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async writeAppConfig (/home/yellowtent/box/src/reverseproxy.js:557:9)
at async Object.configureApp (/home/yellowtent/box/src/reverseproxy.js:574:5)
at async install (/home/yellowtent/box/src/apptask.js:407:5)
May 11 23:06:35 box:taskworker Task took 136.041 seconds
May 11 23:06:35 box:tasks setCompleted - 8146: {"result":null,"error":{"stack":"TypeError: Cannot set properties of undefined (setting 'taskId')\n at makeTaskError (/home/yellowtent/box/src/apptask.js:54:26)\n at run (/home/yellowtent/box/src/apptask.js:814:86)\n at runMicrotasks (<anonymous>)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)","message":"Cannot set properties of undefined (setting 'taskId')"}}
May 11 23:06:35 box:tasks update 8146: {"percent":100,"result":null,"error":{"stack":"TypeError: Cannot set properties of undefined (setting 'taskId')\n at makeTaskError (/home/yellowtent/box/src/apptask.js:54:26)\n at run (/home/yellowtent/box/src/apptask.js:814:86)\n at runMicrotasks (<anonymous>)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)","message":"Cannot set properties of undefined (setting 'taskId')"}}
Cannot set properties of undefined (setting 'taskId')
at makeTaskError (/home/yellowtent/box/src/apptask.js:54:26)
at run (/home/yellowtent/box/src/apptask.js:814:86)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
May 11 23:06:35 IAM initialization complete

[jdaviescoates]

If I go to the url I get this:

You are seeing this page because the DNS record of wordpress-backups.ud.coop is set to this server's IP but Cloudron has no app configured for this domain.

I thought I'd try hitting save in Location and I get this error:

secondaryDomain API_SERVER_DOMAIN is not listed in manifest

It seems something about the new API server domain of the new version is messing up restoring the older version

Help!

[jdaviescoates]

Figured I'd try cloning from one of 1.165.0 backups, same error:

secondaryDomain API_SERVER_DOMAIN is not listed in manifest

Well, yes, that's because in that version is is no API_SERVER_DOMAIN!

Help!

[jdaviescoates]

So, tried restoring the 2.0.0-1 backup instead, because at least that has the API_SERVER_DOMAIN

But that just results in Not Responding, I think because some of the stuff I put in .env.sh was confusing it. So deleted those lines and restarted... and I'm finally back in!

[jdaviescoates]

Now to see if I can get the BackWPUp plugin in WordPress to work again... seems to be. Phew.

But I'm still very confused about the Access and Secret Keys...

Thankfully it turns out I had them saved in Bit/ValutWarden, but they don't appear to be anywhere in Minio itself (there is nothing in the env.sh file), so where/ how is Minio actually storing them?!?

[jdaviescoates]

Anyways, @staff, I'm sorted for now, but I think I've found a bug in that it's currently impossible to restore older versions of Minio as it's expecting there to be an API_SERVER_DOMAIN in the manifest, when of course there isn't because no such thing existed in older versions.

[girish]

@jdaviescoates yes, indeed, this is a bug.

Probably not worth fixing since this was just a special case with this app. i pondered if I just push out the new minio as a totally different app but thought that people might have their existing data there and would be a hassle to migrate.

[jdaviescoates]

@jdaviescoates said in Restoring backups not working - stuck on Restoring:

I think because some of the stuff I put in .env.sh was confusing it.

Also, @staff I think perhaps something should be added to the docs about how the characters $ and # shouldn't be used in password/ keys as that seems to confuse Minio.

I'm pretty sure that's what lead to these errors:

May 11 23:22:58 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:22:58 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:22:58 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:00 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:00 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:23:00 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:02 /app/data/env.sh: line 5: XJxQXBuvsqk4Cei^Y!d9u9iH2NR@WZuzLdix*743P: command not found
May 11 23:23:02 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:02 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:03 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:03 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:03 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:23:03 /app/data/env.sh: line 5: XJxQXBuvsqk4Cei^Y!d9u9iH2NR@WZuzLdix*743P: command not found
May 11 23:23:03 /app/data/env.sh: line 5: WLk2UWtapFJ2fCbzEBM: command not found
May 11 23:23:05 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:05 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:23:05 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:05 /app/data/env.sh: line 5: XJxQXBuvsqk4Cei^Y!d9u9iH2NR@WZuzLdix*743P: command not found
May 11 23:23:08 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:08 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:23:08 /app/data/env.sh: line 5: XJxQXBuvsqk4Cei^Y!d9u9iH2NR@WZuzLdix*743P: command not found
May 11 23:23:08 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:12 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:12 /app/data/env.sh: line 5: XJxQXBuvsqk4Cei^Y!d9u9iH2NR@WZuzLdix*743P: command not found
May 11 23:23:12 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:12 /app/data/env.sh: line 5: WLk2UWtapFJ2fCbzEBM: command not found
May 11 23:23:12 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:23:19 /app/data/env.sh: line 5: Uv: unbound variable
May 11 23:23:19 /app/data/env.sh: line 5: $4: unbound variable
May 11 23:23:19 /app/data/env.sh: line 5: NFrBzhA: unbound variable
May 11 23:23:19 /app/data/env.sh: line 5: XJxQXBuvsqk4Cei^Y!d9u9iH2NR@WZuzLdix*743P: command not found

[jdaviescoates]

@girish well, if you're not going to fix the bug, perhaps at least add a warning somewhere that once you've upgraded it's impossible to restore older backups? Thanks goodness I also had a backup of the latest version otherwise I don't know what I would've done!

[girish]

@jdaviescoates Agreed... Except that we actually made this update 5 months ago. I do wonder how many others have not updated to latest minio. Latest minio package is v2.5.0 (there's been around 15-20 releases since the v2 package).

I think a good lesson for us for next time we do something like this.

[girish]

@jdaviescoates Since it's shell script , maybe you have to single quote things properly. Like so:

export MINIO_ROOT_PASSWORD='minioa#dm$in'

[jdaviescoates]

@girish said in Restoring backups not working - stuck on Restoring:

I do wonder how many others have not updated to latest minio

Given how pervasive the fear of breaking thing by updating is (I used to have to constantly tell people I made Wordpress sites for that the risk of getting hacked by not updating is higher than the risk of breaking something by updating), I'd guess quite a few.

[jdaviescoates]

@girish said in Restoring backups not working - stuck on Restoring:

@jdaviescoates Since it's shell script , maybe you have to single quote things properly. Like so:

export MINIO_ROOT_PASSWORD='minioa#dm$in'

I think the docs need updating to make that clear.

At present they just say:

To change admin credentials, use the File manager to edit the variables MINIO_ROOT_USER and MINIO_ROOT_PASSWORD in /app/data/env.sh

And

1. My env.sh doesn't even have those variables
2. I think you should include a full example like you did above

export MINIO_ROOT_PASSWORD='minioa#dm$in'

Including highlighting that the single quotes are important

Probably also worth adding that this user/ pass combo are also the Access Key and Secret Key too (to avoid similar confusion to what happened with Own Cast)

Thanks!

[girish]

I did a full run of update testing now.

• After upgrade, I was able to login with my previous root credentials.

• After upgrade, indeed, the env variables are not injected. I have put an example in docs now - https://docs.cloudron.io/apps/minio/#admin-credentials . New installs have these variables.

• I was able to reproduce the clone issue. Was easy to fix. I guess this will be useful in case one wants to inspect older backups in the future.

• The restore is a bit more complex - Made an issue https://git.cloudron.io/cloudron/box/-/issues/814