Backup not working since auto-Updating to 8.0.3

[RazielKanos]

Cloudron auoupdated to the 8.0.3 version and since then, every single backup failed with this log entry:

Aug 22 07:39:09 box:taskworker Starting task 3565. Logs are at /home/yellowtent/platformdata/logs/tasks/3565.log
Aug 22 07:39:09 box:tasks update 3565: {"percent":1,"message":"Backing up officeserver.domain.com (1/16)"}
Aug 22 07:39:09 box:tasks update 3565: {"percent":6.2631578947368425,"message":"Snapshotting app officeserver.domain.com"}
Aug 22 07:39:09 box:services backupAddons
Aug 22 07:39:09 box:services backupAddons: backing up ["localstorage","ldap"]
Aug 22 07:39:09 box:backuptask snapshotApp: officeserver.domain.com took 0.003 seconds
Aug 22 07:39:09 box:tasks update 3565: {"percent":6.2631578947368425,"message":"Uploading app snapshot officeserver.domain.com"}
Aug 22 07:39:09 box:shell backup-snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385 /usr/bin/sudo -S -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385 tgz {"localRoot":"/home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385","layout":[]}
Aug 22 07:39:09 box:backupupload Backing up {"localRoot":"/home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385","layout":[]} to snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385
Aug 22 07:39:09 box:backuptask upload: path snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385 format tgz dataLayout {"localRoot":"/home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385","layout":[]}
Aug 22 07:39:09 box:backuptask upload: mount point status is {"state":"active","message":"Mounted"}
Aug 22 07:39:09 box:shell file execArgs: df ["-B1","--output=source,fstype,size,used,avail,pcent,target","/var/backups"]
Aug 22 07:39:09 box:backuptask checkPreconditions: getting disk usage of /home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385
Aug 22 07:39:09 box:shell checkPreconditions execArgs: du ["-Dsb","--exclude=*.lock","--exclude=dovecot.list.index.log.*","/home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385"]
Aug 22 07:39:09 box:backuptask checkPreconditions: total required=38899 available=37746036736
Aug 22 07:39:09 box:backupformat/tgz upload: Uploading {"localRoot":"/home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385","layout":[]} to snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385
Aug 22 07:39:09 box:tasks update 3565: {"percent":6.2631578947368425,"message":"Uploading backup /var/backups/snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385.tar.gz (officeserver.domain.com)"}
Aug 22 07:39:09 box:backupformat/tgz tarPack: processing /home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385
Aug 22 07:39:09 box:backupformat/tgz addToPack: added ./config.json file
Aug 22 07:39:09 box:backupformat/tgz addToPack: added ./data directory
Aug 22 07:39:09 box:backupformat/tgz tarPack: processing /home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385/data
Aug 22 07:39:09 box:backupformat/tgz addToPack: added ./data/coolwsd.xml file
Aug 22 07:39:09 box:backupformat/tgz addToPack: added ./data/fonts directory
Aug 22 07:39:09 box:backupformat/tgz tarPack: processing /home/yellowtent/appsdata/096425e8-f7f4-48e8-b589-10d146cef385/data/fonts
Aug 22 07:39:09 box:backupformat/tgz addToPack: added ./data/fonts/Kontakt.ttf file
Aug 22 07:39:09 box:backupformat/tgz tarPack: pipeline finished: {"startTime":1724305149690,"totalSecs":0,"transferred":9925}
Aug 22 07:39:09 box:backupupload upload completed. error: null
Aug 22 07:39:09 box:backuptask runBackupUpload: result - {"result":""}
Aug 22 07:39:09 box:backuptask uploadAppSnapshot: officeserver.domain.com uploaded to snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385. 0.569 seconds
Aug 22 07:39:09 box:backuptask rotateAppBackup: rotating officeserver.domain.com to path 2024-08-22-053909-108/app_officeserver.domain.com_v1.17.8
Aug 22 07:39:09 box:tasks update 3565: {"percent":6.2631578947368425,"message":"Copying /var/backups/snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385.tar.gz to /var/backups/2024-08-22-053909-108/app_officeserver.domain.com_v1.17.8.tar.gz"}
Aug 22 07:39:09 box:shell copy execArgs: cp ["-al","/var/backups/snapshot/app_096425e8-f7f4-48e8-b589-10d146cef385.tar.gz","/var/backups/2024-08-22-053909-108/app_officeserver.domain.com_v1.17.8.tar.gz"]
Aug 22 07:39:09 box:backuptask copy: copied successfully to 2024-08-22-053909-108/app_officeserver.domain.com_v1.17.8. Took 0.009 seconds
Aug 22 07:39:09 box:backuptask fullBackup: app officeserver.domain.com backup finished. Took 0.596 seconds
Aug 22 07:39:09 box:tasks update 3565: {"percent":6.2631578947368425,"message":"Backing up tw.domain.com (2/16)"}
Aug 22 07:39:09 box:tasks update 3565: {"percent":11.526315789473685,"message":"Snapshotting app tw.domain.com"}
Aug 22 07:39:09 box:services backupAddons
Aug 22 07:39:09 box:services backupAddons: backing up ["localstorage","postgresql","redis","sendmail","oidc","scheduler"]
Aug 22 07:39:09 box:services Backing up postgresql
Aug 22 07:39:09 box:services pipeRequestToFile: connected with status code 500
Aug 22 07:39:09 box:taskworker Task took 0.716 seconds
Aug 22 07:39:09 box:tasks setCompleted - 3565: {"result":null,"error":{"stack":"BoxError: Unexpected response code or HTTP error when piping http://172.18.30.2:3000/databases/db0a0db27fffc44b198cf8ab23c21defd5/backup?access_token=c5f4178efb256ed7c72d8742cf18adc2b921136647ecf6c0c1df11c7df8bb002d19444ff1ce936a92ee4d165ec0cb16ebcd87f8f2f14e7af1ef3bc31f1465d5017f7dc2e8f5ee69ca69db3b80bfb9a0248981e2c40b94f42be2df243b982936e435d835e1f1ca790ccd2531a41cf4d6d5fc65ba59ca5113bffc5648b85c641c6 to /home/yellowtent/appsdata/0a0db27f-ffc4-4b19-8cf8-ab23c21defd5/postgresqldump: status 500\n at ClientRequest.<anonymous> (/home/yellowtent/box/src/services.js:1283:60)\n at ClientRequest.emit (node:events:518:28)\n at HTTPParser.parserOnIncomingClient (node:_http_client:698:27)\n at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)\n at Socket.socketOnData (node:_http_client:540:22)\n at Socket.emit (node:events:518:28)\n at addChunk (node:internal/streams/readable:559:12)\n at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)\n at Readable.push (node:internal/streams/readable:390:5)\n at TCP.onStreamRead (node:internal/stream_base_commons:190:23)","name":"BoxError","reason":"Addons Error","details":{},"message":"Unexpected response code or HTTP error when piping http://172.18.30.2:3000/databases/db0a0db27fffc44b198cf8ab23c21defd5/backup?access_token=c5f4178efb256ed7c72d8742cf18adc2b921136647ecf6c0c1df11c7df8bb002d19444ff1ce936a92ee4d165ec0cb16ebcd87f8f2f14e7af1ef3bc31f1465d5017f7dc2e8f5ee69ca69db3b80bfb9a0248981e2c40b94f42be2df243b982936e435d835e1f1ca790ccd2531a41cf4d6d5fc65ba59ca5113bffc5648b85c641c6 to /home/yellowtent/appsdata/0a0db27f-ffc4-4b19-8cf8-ab23c21defd5/postgresqldump: status 500"}}
Aug 22 07:39:09 box:tasks update 3565: {"percent":100,"result":null,"error":{"stack":"BoxError: Unexpected response code or HTTP error when piping http://172.18.30.2:3000/databases/db0a0db27fffc44b198cf8ab23c21defd5/backup?access_token=c5f4178efb256ed7c72d8742cf18adc2b921136647ecf6c0c1df11c7df8bb002d19444ff1ce936a92ee4d165ec0cb16ebcd87f8f2f14e7af1ef3bc31f1465d5017f7dc2e8f5ee69ca69db3b80bfb9a0248981e2c40b94f42be2df243b982936e435d835e1f1ca790ccd2531a41cf4d6d5fc65ba59ca5113bffc5648b85c641c6 to /home/yellowtent/appsdata/0a0db27f-ffc4-4b19-8cf8-ab23c21defd5/postgresqldump: status 500\n at ClientRequest.<anonymous> (/home/yellowtent/box/src/services.js:1283:60)\n at ClientRequest.emit (node:events:518:28)\n at HTTPParser.parserOnIncomingClient (node:_http_client:698:27)\n at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)\n at Socket.socketOnData (node:_http_client:540:22)\n at Socket.emit (node:events:518:28)\n at addChunk (node:internal/streams/readable:559:12)\n at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)\n at Readable.push (node:internal/streams/readable:390:5)\n at TCP.onStreamRead (node:internal/stream_base_commons:190:23)","name":"BoxError","reason":"Addons Error","details":{},"message":"Unexpected response code or HTTP error when piping http://172.18.30.2:3000/databases/db0a0db27fffc44b198cf8ab23c21defd5/backup?access_token=c5f4178efb256ed7c72d8742cf18adc2b921136647ecf6c0c1df11c7df8bb002d19444ff1ce936a92ee4d165ec0cb16ebcd87f8f2f14e7af1ef3bc31f1465d5017f7dc2e8f5ee69ca69db3b80bfb9a0248981e2c40b94f42be2df243b982936e435d835e1f1ca790ccd2531a41cf4d6d5fc65ba59ca5113bffc5648b85c641c6 to /home/yellowtent/appsdata/0a0db27f-ffc4-4b19-8cf8-ab23c21defd5/postgresqldump: status 500"}}
[no timestamp]  Unexpected response code or HTTP error when piping http://172.18.30.2:3000/databases/db0a0db27fffc44b198cf8ab23c21defd5/backup?access_token=c5f4178efb256ed7c72d8742cf18adc2b921136647ecf6c0c1df11c7df8bb002d19444ff1ce936a92ee4d165ec0cb16ebcd87f8f2f14e7af1ef3bc31f1465d5017f7dc2e8f5ee69ca69db3b80bfb9a0248981e2c40b94f42be2df243b982936e435d835e1f1ca790ccd2531a41cf4d6d5fc65ba59ca5113bffc5648b85c641c6 to /home/yellowtent/appsdata/0a0db27f-ffc4-4b19-8cf8-ab23c21defd5/postgresqldump: status 500
[no timestamp]  at ClientRequest.<anonymous> (/home/yellowtent/box/src/services.js:1283:60)
[no timestamp]  at ClientRequest.emit (node:events:518:28)
[no timestamp]  at HTTPParser.parserOnIncomingClient (node:_http_client:698:27)
[no timestamp]  at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
[no timestamp]  at Socket.socketOnData (node:_http_client:540:22)
[no timestamp]  at Socket.emit (node:events:518:28)
[no timestamp]  at addChunk (node:internal/streams/readable:559:12)
[no timestamp]  at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
[no timestamp]  at Readable.push (node:internal/streams/readable:390:5)
[no timestamp]  at TCP.onStreamRead (node:internal/stream_base_commons:190:23)

I also noticed that my linkding gave me a error 500, and when i wanted to restore it from a backup i got this error:
Addons Error: Error setting up postgresql. Status code: 500 message: connect ECONNREFUSED ::1:5432

Same Error Message with CHatwoot

[RazielKanos]

Nope, my immich also isn't working - seems all my apps are broken since that update

[RazielKanos]

I noticed that postgresql constantly is trying to restart, so i stopped the service and went from 756 MB up to 2 GB. It started again and now is stitting at 53% usage. MY apps are working again, only chatwoot still is unresponding, but i am updating it right now, and will try to run a backup once it's finished

[RazielKanos]

Chatwoot is running after the update too, and the beackups are working as well

Solution:

Check services
added more ram to database usage

Topic can be closed

[humptydumpty]

For future reference, paste your code using the code </> button. Your post was a loooong scroll on mobile.

[RazielKanos]

@humptydumpty updated ^^