Backups keep failing

[nebulon]

Those are immich errors, not related to any Cloudron backup. Looks like immich added its own database backup job by now which will not work on Cloudron and we have to disable this in the package. But either way you can ignore those. What is the status on your Cloudron backup of the app?

[OrezBodajoy]

No new immich backup. Right now the app is marked as Error and on the backup tab the 'Create backup' button is greyed-out.
In the notifications I also get a backup task crashed notification.
Here are the logs for both immich and that general backup task crashed.

https://upload.disroot.org/r/nuvYocg7#xGyeIOHzUekVeQhAC3Aup1vR7/SZqPepg3/aC10aSW0=
https://upload.disroot.org/r/9MFpVBdn#B9BzBZ8buWuNh0SPq6j605GBQcCqIedo0G/jI2J956g=

Please have a look at the logs.
Thanks!

[nebulon]

For a start, the next immich package version has the built-in database backups disabled, as they are of no use on Cloudron and also try to write into read-only storage.

To get your app up and running again, please hit the "Retry task" in the repair tab of that apps configure section.

[nebulon]

Looking further at the logs, it seems at least that last run still hit the EPIPE error, where the uploading was cancelled while trying to upload a .mp4 file. This could either be a memory issue on your server or also the remote storage terminated the connection prematurely.

[OrezBodajoy]

I have changed a bit the backup parameters in the UI:

• memory limit to 9 GiB
• upload part size 64 MiB
• concurrency items 10

I now have this error:

Feb 04 13:48:29 box:storage/s3 Upload progress: {"loaded":62003204,"total":62003204,"part":1,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/e4LlBfnRs2P83jBNmVfv-h8S-snRVHnPmaS4dtZW2vY/ePMwmyFhRxqFNN7QBJzlCWOFsK-Qvsw42NbUryxdEiU/7NKck9Wkh-+bm1O0jmO6SyT3eqB1rwGEE5nHJje5Vc2nihu8hYC7UyF+WvX4eM5wpaPa04Cp9mm0g9jyU+WHPQ"}
Feb 04 13:48:32 box:backupformat/rsync sync: processing task: {"operation":"add","path":"data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f6/e3/f6e37991-d39d-4bab-9317-afca881eda91.mp4","reason":"new","position":425}
Feb 04 13:48:32 box:storage/s3 Upload finished. {"ETag":"\"96b07711e9e9cd6313ae1d3bf2eab89d\"","VersionId":"4_z30c7662aa70ee25089da0419_f112b8445e8479200_d20250204_m124750_c003_v0312024_t0049_u01738673270973","Location":"https://cloudron-backup-january-february.s3.eu-central-003.backblazeb2.com/snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k%2B5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg%2BwW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ%2BUS7GhiAPkvxud%2BGhx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/0KPhbUuBjncr7PGqgZwzB4fmP2RhEubtj8cyGzIjOgc/P1wybT781lXr7wdAaxpbhPW9khj0bqxBeCNwK00FB18/xY3HW0Fc9c3GoFxrS7ipncSqa80AEMVc%2B63z%2BGW%2BIIMmG24hCz8ddQgvpP45fpFhQ9dRfXTcMloEea9KhYFzYQ","key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/0KPhbUuBjncr7PGqgZwzB4fmP2RhEubtj8cyGzIjOgc/P1wybT781lXr7wdAaxpbhPW9khj0bqxBeCNwK00FB18/xY3HW0Fc9c3GoFxrS7ipncSqa80AEMVc+63z+GW+IIMmG24hCz8ddQgvpP45fpFhQ9dRfXTcMloEea9KhYFzYQ","Key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/0KPhbUuBjncr7PGqgZwzB4fmP2RhEubtj8cyGzIjOgc/P1wybT781lXr7wdAaxpbhPW9khj0bqxBeCNwK00FB18/xY3HW0Fc9c3GoFxrS7ipncSqa80AEMVc+63z+GW+IIMmG24hCz8ddQgvpP45fpFhQ9dRfXTcMloEea9KhYFzYQ","Bucket":"cloudron-backup-january-february"}
Feb 04 13:48:32 box:backupformat/rsync Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f6/e3/f6e37991-d39d-4bab-9317-afca881eda91.mp4 position 425 try 1
Feb 04 13:48:32 box:tasks update 3016: {"percent":30,"message":"Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f6/e3/f6e37991-d39d-4bab-9317-afca881eda91.mp4 (photos.mysuper.website)"}
Feb 04 13:48:34 box:storage/s3 Upload progress: {"loaded":12934596,"total":12934596,"part":1,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/DBIzrCxVqVk3kzOgs2UfGVfVwvfHWqLVm-Fvk6CA-e4/Xz7ApaTu7-QxV5ai4eLprhxdYTI9yssL86kbCdFi9FM/IV6dab6-MyVWBPeF8FJ8ItaEjR8kqXcTxI5pmszMhqFzG2uEA053FqFdR34oPN77RWkwkhadsc2SqtEYbSSlvw"}
Feb 04 13:48:37 box:storage/s3 Upload progress: {"loaded":24622868,"total":24622868,"part":1,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/8PCmzF3EVYPE+WncvS7A7V4qP9u3qm3s6NSg0bNwB4s/XaenUbcaa72K2xvF3+wvbOb8sS2ACmIApKq47zKPb9I/RKfaJr9nuoGi6m7KLX-bABpNePzu2pgw0qXGosmKk1kzAzwZ9SKPxuVP8KnXvDpPHZRdrwWeSAQoKVE4PX1NUA"}
Feb 04 13:48:38 box:backupformat/rsync sync: processing task: {"operation":"add","path":"data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/14/f7145cfc-2306-45f1-92f7-5a19add3aa9c.mp4","reason":"new","position":426}
Feb 04 13:48:38 box:storage/s3 Upload finished. {"ETag":"\"a63eb3273bd8c6af9c8be8cdb8b3411c\"","VersionId":"4_z30c7662aa70ee25089da0419_f111b63a9bf79370d_d20250204_m124018_c003_v0312024_t0006_u01738672818986","Location":"https://cloudron-backup-january-february.s3.eu-central-003.backblazeb2.com/snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k%2B5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg%2BwW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ%2BUS7GhiAPkvxud%2BGhx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/e4LlBfnRs2P83jBNmVfv-h8S-snRVHnPmaS4dtZW2vY/ePMwmyFhRxqFNN7QBJzlCWOFsK-Qvsw42NbUryxdEiU/7NKck9Wkh-%2Bbm1O0jmO6SyT3eqB1rwGEE5nHJje5Vc2nihu8hYC7UyF%2BWvX4eM5wpaPa04Cp9mm0g9jyU%2BWHPQ","key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/e4LlBfnRs2P83jBNmVfv-h8S-snRVHnPmaS4dtZW2vY/ePMwmyFhRxqFNN7QBJzlCWOFsK-Qvsw42NbUryxdEiU/7NKck9Wkh-+bm1O0jmO6SyT3eqB1rwGEE5nHJje5Vc2nihu8hYC7UyF+WvX4eM5wpaPa04Cp9mm0g9jyU+WHPQ","Key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/e4LlBfnRs2P83jBNmVfv-h8S-snRVHnPmaS4dtZW2vY/ePMwmyFhRxqFNN7QBJzlCWOFsK-Qvsw42NbUryxdEiU/7NKck9Wkh-+bm1O0jmO6SyT3eqB1rwGEE5nHJje5Vc2nihu8hYC7UyF+WvX4eM5wpaPa04Cp9mm0g9jyU+WHPQ","Bucket":"cloudron-backup-january-february"}
Feb 04 13:48:38 box:backupformat/rsync Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/14/f7145cfc-2306-45f1-92f7-5a19add3aa9c.mp4 position 426 try 1
Feb 04 13:48:38 box:tasks update 3016: {"percent":30,"message":"Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/14/f7145cfc-2306-45f1-92f7-5a19add3aa9c.mp4 (photos.mysuper.website)"}
Feb 04 13:48:43 box:storage/s3 Upload finished. {"ETag":"\"e8f727fbe3017a736863acd3e005920f\"","VersionId":"4_z30c7662aa70ee25089da0419_f102a9d6210f3b68d_d20250204_m124644_c003_v0312028_t0032_u01738673204217","Location":"https://cloudron-backup-january-february.s3.eu-central-003.backblazeb2.com/snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k%2B5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg%2BwW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ%2BUS7GhiAPkvxud%2BGhx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/DBIzrCxVqVk3kzOgs2UfGVfVwvfHWqLVm-Fvk6CA-e4/Xz7ApaTu7-QxV5ai4eLprhxdYTI9yssL86kbCdFi9FM/IV6dab6-MyVWBPeF8FJ8ItaEjR8kqXcTxI5pmszMhqFzG2uEA053FqFdR34oPN77RWkwkhadsc2SqtEYbSSlvw","key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/DBIzrCxVqVk3kzOgs2UfGVfVwvfHWqLVm-Fvk6CA-e4/Xz7ApaTu7-QxV5ai4eLprhxdYTI9yssL86kbCdFi9FM/IV6dab6-MyVWBPeF8FJ8ItaEjR8kqXcTxI5pmszMhqFzG2uEA053FqFdR34oPN77RWkwkhadsc2SqtEYbSSlvw","Key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/DBIzrCxVqVk3kzOgs2UfGVfVwvfHWqLVm-Fvk6CA-e4/Xz7ApaTu7-QxV5ai4eLprhxdYTI9yssL86kbCdFi9FM/IV6dab6-MyVWBPeF8FJ8ItaEjR8kqXcTxI5pmszMhqFzG2uEA053FqFdR34oPN77RWkwkhadsc2SqtEYbSSlvw","Bucket":"cloudron-backup-january-february"}
Feb 04 13:48:43 box:backupformat/rsync Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/70/f770093e-d633-4553-8e18-76928a64abca.mp4 position 427 try 1
Feb 04 13:48:43 box:backupformat/rsync sync: processing task: {"operation":"add","path":"data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/70/f770093e-d633-4553-8e18-76928a64abca.mp4","reason":"new","position":427}
Feb 04 13:48:45 box:storage/s3 Upload finished. {"ETag":"\"fef4d9e41a7b0487ef85cde77de337c7\"","VersionId":"4_z30c7662aa70ee25089da0419_f105b54dc655def0e_d20250204_m124514_c003_v0312025_t0014_u01738673114140","Location":"https://cloudron-backup-january-february.s3.eu-central-003.backblazeb2.com/snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k%2B5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg%2BwW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ%2BUS7GhiAPkvxud%2BGhx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/8PCmzF3EVYPE%2BWncvS7A7V4qP9u3qm3s6NSg0bNwB4s/XaenUbcaa72K2xvF3%2BwvbOb8sS2ACmIApKq47zKPb9I/RKfaJr9nuoGi6m7KLX-bABpNePzu2pgw0qXGosmKk1kzAzwZ9SKPxuVP8KnXvDpPHZRdrwWeSAQoKVE4PX1NUA","key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/8PCmzF3EVYPE+WncvS7A7V4qP9u3qm3s6NSg0bNwB4s/XaenUbcaa72K2xvF3+wvbOb8sS2ACmIApKq47zKPb9I/RKfaJr9nuoGi6m7KLX-bABpNePzu2pgw0qXGosmKk1kzAzwZ9SKPxuVP8KnXvDpPHZRdrwWeSAQoKVE4PX1NUA","Key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/8PCmzF3EVYPE+WncvS7A7V4qP9u3qm3s6NSg0bNwB4s/XaenUbcaa72K2xvF3+wvbOb8sS2ACmIApKq47zKPb9I/RKfaJr9nuoGi6m7KLX-bABpNePzu2pgw0qXGosmKk1kzAzwZ9SKPxuVP8KnXvDpPHZRdrwWeSAQoKVE4PX1NUA","Bucket":"cloudron-backup-january-february"}
Feb 04 13:48:45 box:backupformat/rsync sync: processing task: {"operation":"add","path":"data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/d8/f7d8868b-8af8-489d-a6e8-3ba76e06f810.mp4","reason":"new","position":428}
Feb 04 13:48:45 box:backupformat/rsync Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/d8/f7d8868b-8af8-489d-a6e8-3ba76e06f810.mp4 position 428 try 1
Feb 04 13:48:45 box:tasks update 3016: {"percent":30,"message":"Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f7/d8/f7d8868b-8af8-489d-a6e8-3ba76e06f810.mp4 (photos.mysuper.website)"}
Feb 04 13:48:48 box:storage/s3 Upload progress: {"loaded":83903444,"total":83903444,"part":1,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/7TW2OB1TBrQ57XB3jzn8HgA7Nke74zrlTG7lc0704xk/w1cW6moKv9lYkZgZDxUkRN4Zxl8myZ-CEvE3Ye-HQhg/gb9sGl1g7pgYmuGbcL83+N073h+YeIRnZK2s9dbPK5wxKEhNFw59MZiZqfU2f9QieTqgP-tXt9Njcir3yGQYEg"}
Feb 04 13:48:56 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:48:56 zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg%2BwW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ%2BUS7GhiAPkvxud%2BGhx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/7TW2OB1TBrQ57XB3jzn8HgA7Nke74zrlTG7lc0704xk/w1cW6moKv9lYkZgZDxUkRN4Zxl8myZ-CEvE3Ye-HQhg/gb9sGl1g7pgYmuGbcL83%2BN073h%2BYeIRnZK2s9dbPK5wxKEhNFw59MZiZqfU2f9QieTqgP-tXt9Njcir3yGQYEg","Bucket":"cloudron-backup-january-february","Key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/7TW2OB1TBrQ57XB3jzn8HgA7Nke74zrlTG7lc0704xk/w1cW6moKv9lYkZgZDxUkRN4Zxl8myZ-CEvE3Ye-HQhg/gb9sGl1g7pgYmuGbcL83+N073h+YeIRnZK2s9dbPK5wxKEhNFw59MZiZqfU2f9QieTqgP-tXt9Njcir3yGQYEg","ETag":"\"de497871d785fd1c59918d801ccfa714-2\""}
Feb 04 13:48:56 box:backupformat/rsync Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f8/f3/f8f30130-24cd-47e8-872a-c804bd9ca9da.mp4 position 429 try 1
Feb 04 13:48:56 box:backupformat/rsync sync: processing task: {"operation":"add","path":"data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f8/f3/f8f30130-24cd-47e8-872a-c804bd9ca9da.mp4","reason":"new","position":429}
Feb 04 13:49:06 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:49:11 box:storage/s3 Upload progress: {"loaded":51287460,"total":118396324,"part":2,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/JXzM88Uqx7ZsSnmpRN00J7pjTkogYPeJj9sRGLylY-Y/lCpIzgRkjCJJlzwTqL5scjzLiKZZcSHPgHnseJC+9VU/xzreWsN82y+-8efk7r1FwykGI9qCIdoC1+WVWtIjKuolkKXYiFi4nMAPlEa4HbLqzGkhp4PA7nDfRQZSvpQgNA"}
Feb 04 13:49:12 13:M 04 Feb 2025 12:49:12.044 * 10 changes in 300 seconds. Saving...
Feb 04 13:49:12 13:M 04 Feb 2025 12:49:12.046 * Background saving started by pid 248
Feb 04 13:49:12 13:M 04 Feb 2025 12:49:12.146 * Background saving terminated with success
Feb 04 13:49:12 248:C 04 Feb 2025 12:49:12.129 * DB saved on disk
Feb 04 13:49:12 248:C 04 Feb 2025 12:49:12.130 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 13:49:16 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:49:26 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:49:36 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:49:46 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:49:56 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:50:06 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:50:12 box:storage/s3 Upload progress: {"loaded":79710836,"total":146819700,"part":1,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/LQs0pg3EACKpxLfIoyhm3-pwf2CjUC0Qmr45cpZ0XtY/zRON9tC4HPaxKx9vNEkZkXGSyTtIBEVsHwJTKqt6EAI/zSGMZ1+pEr5om7CkzelvYADDA4IPqQHHBv18fQYz-AzjLcjdhD8N9ce0c7KaGaYbYwyPXwq5IcGX07SjbJQWzw"}
Feb 04 13:50:16 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:50:26 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:50:36 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:50:36 box:storage/s3 Upload progress: {"loaded":21172484,"total":21172484,"part":1,"key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/SOlrDfP9m6u2e4I5p44PIDh8HSg0IPGy2LPaDZ72PEo/l0+gncIhXvicAKKVNFY-njrXTET921WkntYTifcK7yE/d0AO9QuFyiYOpalIzyM0LA9WAyPtS9faM719jCVO5oR81AAyJiG9gzNp4pwWaUjy+IEnSGqo17KBKZjlQcIp7Q"}
Feb 04 13:50:39 box:backupformat/rsync Adding data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f9/84/f984597c-729e-406f-ad68-73275081a91b.mp4 position 430 try 1
Feb 04 13:50:39 box:backupformat/rsync sync: processing task: {"operation":"add","path":"data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f9/84/f984597c-729e-406f-ad68-73275081a91b.mp4","reason":"new","position":430}
Feb 04 13:50:39 box:storage/s3 Upload finished. {"ETag":"\"e8db66fe56ced58e8bccbf20014a6b92\"","VersionId":"4_z30c7662aa70ee25089da0419_f1064034b6c53492b_d20250204_m124857_c003_v0312028_t0047_u01738673337237","Location":"https://cloudron-backup-january-february.s3.eu-central-003.backblazeb2.com/snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k%2B5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg%2BwW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ%2BUS7GhiAPkvxud%2BGhx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/SOlrDfP9m6u2e4I5p44PIDh8HSg0IPGy2LPaDZ72PEo/l0%2BgncIhXvicAKKVNFY-njrXTET921WkntYTifcK7yE/d0AO9QuFyiYOpalIzyM0LA9WAyPtS9faM719jCVO5oR81AAyJiG9gzNp4pwWaUjy%2BIEnSGqo17KBKZjlQcIp7Q","key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/SOlrDfP9m6u2e4I5p44PIDh8HSg0IPGy2LPaDZ72PEo/l0+gncIhXvicAKKVNFY-njrXTET921WkntYTifcK7yE/d0AO9QuFyiYOpalIzyM0LA9WAyPtS9faM719jCVO5oR81AAyJiG9gzNp4pwWaUjy+IEnSGqo17KBKZjlQcIp7Q","Key":"snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076/By8169k+5t6ApU5J9RRxIHnhEdUt9ZLcwMdOzzg4zaY/oEtU3dvIl7fyle9hukEV1y6EC9fxUgsZDIZvg+wW5Qg/axJfbCsBMvfcT8e8Uy-FZW8CPjKDUUBAaFaBe4bx9jg/Xe4iKBFLF9isOK0zcwdxWiDMGoCzdnvmJ+US7GhiAPkvxud+Ghx0RTG3jfbRrboc-texm4iEKjFt9iuZo2WA3Q/SOlrDfP9m6u2e4I5p44PIDh8HSg0IPGy2LPaDZ72PEo/l0+gncIhXvicAKKVNFY-njrXTET921WkntYTifcK7yE/d0AO9QuFyiYOpalIzyM0LA9WAyPtS9faM719jCVO5oR81AAyJiG9gzNp4pwWaUjy+IEnSGqo17KBKZjlQcIp7Q","Bucket":"cloudron-backup-january-february"}
Feb 04 13:50:45 box:backupformat/rsync Attempt 1 failed. Will retry: Upload error: code: 408 message: null
Feb 04 13:50:46 box:backupformat/rsync Attempt 1 failed. Will retry: Upload error: code: 408 message: null
Feb 04 13:50:46 box:tasks update 3016: {"percent":30,"message":"Uploading /home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076/data/upload/encoded-video/177472c9-60f1-4746-a2c2-fd947abb8cca/f5/56/f556116d-2ae2-4a13-bd53-bf315134c7ae.mp4: 256M@0MBps (photos.mysuper.website)"}
Feb 04 13:50:46 box:backupformat/rsync Attempt 1 failed. Will retry: Upload error: code: 408 message: null
Feb 04 13:50:46 box:backupformat/rsync Attempt 1 failed. Will retry: Upload error: code: 408 message: null
Feb 04 13:50:46 box:shell backuptask: /usr/bin/sudo -S -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_14dd2bbe-1d61-4f3d-91d3-4af16e366076 rsync {"localRoot":"/home/yellowtent/appsdata/14dd2bbe-1d61-4f3d-91d3-4af16e366076","layout":[]} errored BoxError: backuptask exited with code 1 signal null
Feb 04 13:50:46 box:backuptask runBackupUpload: backuptask crashed BoxError: backuptask exited with code 1 signal null
Feb 04 13:50:46 box:apptask run: app error for state pending_backup: BoxError: Backuptask crashed at runBackupUpload (/home/yellowtent/box/src/backuptask.js:170:15) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:369:5) at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:391:5) at async Object.backupApp (/home/yellowtent/box/src/backuptask.js:329:12) at async backup (/home/yellowtent/box/src/apptask.js:400:22) { reason: 'Internal Error', details: {} }
Feb 04 13:50:46 box:taskworker Task took 12148.089 seconds
Feb 04 13:50:46 box:tasks setCompleted - 3016: {"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:170:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:369:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:391:5)\n at async Object.backupApp (/home/yellowtent/box/src/backuptask.js:329:12)\n at async backup (/home/yellowtent/box/src/apptask.js:400:22)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}}
Feb 04 13:50:46 box:tasks update 3016: {"percent":100,"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:170:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:369:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:391:5)\n at async Object.backupApp (/home/yellowtent/box/src/backuptask.js:329:12)\n at async backup (/home/yellowtent/box/src/apptask.js:400:22)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}}
Feb 04 13:54:13 13:M 04 Feb 2025 12:54:13.044 * 10 changes in 300 seconds. Saving...
Feb 04 13:54:13 13:M 04 Feb 2025 12:54:13.046 * Background saving started by pid 249
Feb 04 13:54:13 13:M 04 Feb 2025 12:54:13.147 * Background saving terminated with success
Feb 04 13:54:13 249:C 04 Feb 2025 12:54:13.137 * DB saved on disk
Feb 04 13:54:13 249:C 04 Feb 2025 12:54:13.139 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 13:59:14 13:M 04 Feb 2025 12:59:14.020 * 10 changes in 300 seconds. Saving...
Feb 04 13:59:14 13:M 04 Feb 2025 12:59:14.022 * Background saving started by pid 250
Feb 04 13:59:14 13:M 04 Feb 2025 12:59:14.223 * Background saving terminated with success
Feb 04 13:59:14 250:C 04 Feb 2025 12:59:14.124 * DB saved on disk
Feb 04 13:59:14 250:C 04 Feb 2025 12:59:14.125 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:04:15 13:M 04 Feb 2025 13:04:15.094 * 10 changes in 300 seconds. Saving...
Feb 04 14:04:15 13:M 04 Feb 2025 13:04:15.097 * Background saving started by pid 251
Feb 04 14:04:15 13:M 04 Feb 2025 13:04:15.198 * Background saving terminated with success
Feb 04 14:04:15 251:C 04 Feb 2025 13:04:15.188 * DB saved on disk
Feb 04 14:04:15 251:C 04 Feb 2025 13:04:15.189 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:09:16 13:M 04 Feb 2025 13:09:16.081 * 10 changes in 300 seconds. Saving...
Feb 04 14:09:16 13:M 04 Feb 2025 13:09:16.082 * Background saving started by pid 252
Feb 04 14:09:16 13:M 04 Feb 2025 13:09:16.285 * Background saving terminated with success
Feb 04 14:09:16 252:C 04 Feb 2025 13:09:16.184 * DB saved on disk
Feb 04 14:09:16 252:C 04 Feb 2025 13:09:16.185 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:14:17 13:M 04 Feb 2025 13:14:17.059 * 10 changes in 300 seconds. Saving...
Feb 04 14:14:17 13:M 04 Feb 2025 13:14:17.061 * Background saving started by pid 253
Feb 04 14:14:17 13:M 04 Feb 2025 13:14:17.262 * Background saving terminated with success
Feb 04 14:14:17 253:C 04 Feb 2025 13:14:17.160 * DB saved on disk
Feb 04 14:14:17 253:C 04 Feb 2025 13:14:17.162 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:19:18 13:M 04 Feb 2025 13:19:18.096 * 10 changes in 300 seconds. Saving...
Feb 04 14:19:18 13:M 04 Feb 2025 13:19:18.098 * Background saving started by pid 254
Feb 04 14:19:18 13:M 04 Feb 2025 13:19:18.198 * Background saving terminated with success
Feb 04 14:19:18 254:C 04 Feb 2025 13:19:18.175 * DB saved on disk
Feb 04 14:19:18 254:C 04 Feb 2025 13:19:18.177 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:24:19 13:M 04 Feb 2025 13:24:19.048 * 10 changes in 300 seconds. Saving...
Feb 04 14:24:19 13:M 04 Feb 2025 13:24:19.049 * Background saving started by pid 255
Feb 04 14:24:19 13:M 04 Feb 2025 13:24:19.149 * Background saving terminated with success
Feb 04 14:24:19 255:C 04 Feb 2025 13:24:19.142 * DB saved on disk
Feb 04 14:24:19 255:C 04 Feb 2025 13:24:19.144 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:29:20 13:M 04 Feb 2025 13:29:20.087 * 10 changes in 300 seconds. Saving...
Feb 04 14:29:20 13:M 04 Feb 2025 13:29:20.089 * Background saving started by pid 256
Feb 04 14:29:20 13:M 04 Feb 2025 13:29:20.190 * Background saving terminated with success
Feb 04 14:29:20 256:C 04 Feb 2025 13:29:20.163 * DB saved on disk
Feb 04 14:29:20 256:C 04 Feb 2025 13:29:20.165 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:34:21 13:M 04 Feb 2025 13:34:21.052 * 10 changes in 300 seconds. Saving...
Feb 04 14:34:21 13:M 04 Feb 2025 13:34:21.053 * Background saving started by pid 257
Feb 04 14:34:21 13:M 04 Feb 2025 13:34:21.154 * Background saving terminated with success
Feb 04 14:34:21 257:C 04 Feb 2025 13:34:21.134 * DB saved on disk
Feb 04 14:34:21 257:C 04 Feb 2025 13:34:21.136 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Feb 04 14:34:21 408: null
Feb 04 14:34:21 BoxError: Backuptask crashed
Feb 04 14:34:21 Node.js v20.18.0
Feb 04 14:34:21 }
Feb 04 14:34:21 }
Feb 04 14:34:21 }

Thanks!

[joseph]

It's the dreaded 408 error - request time out . But I also notice @0Mbps which I think was fixed in 8.2 .

@OrezBodajoy which version of Cloudron are you on?

[OrezBodajoy]

Platform versionv8.2.3 (Ubuntu 22.04.4 LTS)
I will try updating to 8.2.4

Would it work better if I use another backup method than backblaze?

[joseph]

@OrezBodajoy 8.2.4 has nothing backup related, so I think that won't resolve this issue. If you can use something other than B2, it will help atleast to isolate if this is a backblaze issue.

There's been many reports on this forum already about B2 having network issues. Maybe they throttle traffic after some upload sizes and maybe Cloudron code has to be smarter somehow.

[OrezBodajoy]

I am trying now with AWS.
I didn't get an epipe error but the "Task 3061 timed out" after 1h exactly.
My photo album is quite big and I don't expect it to be uploaded within an hour. Is there a way to increase this timeout limit for the backup task? I don't see anything about that in the advanced section of the backup configuration.

Thanks

[nebulon]

Can you share the logs for that also? There shouldn't be any 1h cutoff timeout set by Cloudron itself, so likely this is some other root cause.