<![CDATA[error while backing up]]>Any idea? Maybe a bug?

Dec 05 23:12:02 box:backuptask rotateMailBackup: rotating to 2024-12-05-220000-794/mail_v8.1.0
Dec 05 23:12:02 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying with concurrency of 10"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying files from 0-1"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying (multipart) snapshot/mail.tar.gz"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 1 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=0-1073741823"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 2 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=1073741824-2147483647"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 3 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=2147483648-3221225471"}
Dec 05 23:13:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:13:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:13:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:14:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 2 - Etag: 77e882104ff405cdbf8d091053169ca7"}
Dec 05 23:14:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 4 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=3221225472-4294967295"}
Dec 05 23:14:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (2) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:14:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (2) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:15:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:15:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (3) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:15:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (3) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:16:43 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (2) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:17:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (4) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:17:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (4) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:18:03 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (3) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:18:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (5) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:18:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (5) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:19:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (4) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:19:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (6) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:19:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (6) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:20:43 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (5) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:21:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (7) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:21:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (7) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:22:03 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (6) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:22:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (8) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:22:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (8) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:23:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (7) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:23:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (9) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:23:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (9) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:24:41 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 4 - Etag: a2132197a4d42261637fc197ed066b13"}
Dec 05 23:24:41 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 5 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=4294967296-4776642292"}
Dec 05 23:25:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 1 - Etag: f42130621cb6ce886373fc94e97a0940"}
Dec 05 23:25:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (10) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:25:09 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 5 - Etag: e60e0408e3be489be7986b759a8f3606"}
Dec 05 23:26:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (11) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:26:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Aborting multipart copy of snapshot/mail.tar.gz"}
Dec 05 23:26:28 box:storage/s3 copy: s3 copy error when copying snapshot/mail.tar.gz: 504: null
Dec 05 23:26:28 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copied 1 files with error: BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null"}
Dec 05 23:26:28 box:backuptask copy: copied to 2024-12-05-220000-794/mail_v8.1.0 errored. error: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null
Dec 05 23:26:28 box:taskworker Task took 1587.812 seconds
Dec 05 23:26:28 box:tasks setCompleted - 2014: {"result":null,"error":{"stack":"BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null\n at done (/home/yellowtent/box/src/storage/s3.js:333:48)\n at Response.<anonymous> (/home/yellowtent/box/src/storage/s3.js:409:71)\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:367:18)\n at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)\n at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)","name":"BoxError","reason":"External Error","details":{},"message":"Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null"}}
Dec 05 23:26:28 box:tasks update 2014: {"percent":100,"result":null,"error":{"stack":"BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null\n at done (/home/yellowtent/box/src/storage/s3.js:333:48)\n at Response.<anonymous> (/home/yellowtent/box/src/storage/s3.js:409:71)\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:367:18)\n at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)\n at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)","name":"BoxError","reason":"External Error","details":{},"message":"Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null"}}
Dec 05 23:26:28 BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null
Dec 05 23:26:28 at done (/home/yellowtent/box/src/storage/s3.js:333:48)
Dec 05 23:26:28 at Response.<anonymous> (/home/yellowtent/box/src/storage/s3.js:409:71)
Dec 05 23:26:28 llowtent/box/node_modules/aws-sdk/lib/request.js:367:18)
Dec 05 23:26:28 at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
Dec 05 23:26:28 at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
Dec 05 23:26:28 at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)
Dec 05 23:26:28 at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
Dec 05 23:26:28 at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
Dec 05 23:26:28 at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
Dec 05 23:26:28 at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)

29935785-1adc-4a63-b002-a041f38a0c64-image.png

]]>https://forum.cloudron.io/topic/12909/error-while-backing-upRSS for NodeFri, 17 Apr 2026 03:23:00 GMTFri, 06 Dec 2024 16:39:09 GMT60<![CDATA[Reply to error while backing up on Tue, 10 Dec 2024 21:37:38 GMT]]>It was happening across multiple servers at the same time yesterday. Looks to be resolved now.

]]>https://forum.cloudron.io/post/98055https://forum.cloudron.io/post/98055Tue, 10 Dec 2024 21:37:38 GMT<![CDATA[Reply to error while backing up on Tue, 10 Dec 2024 21:34:53 GMT]]>@Dave-Swift this happens all the time? If it's sporadic, then I suspect this is just a Hetzner issue (are they still in beta?). Curiously, I am not finding any announcement related to the stability/availability .

]]>https://forum.cloudron.io/post/98053https://forum.cloudron.io/post/98053Tue, 10 Dec 2024 21:34:53 GMT<![CDATA[Reply to error while backing up on Tue, 10 Dec 2024 02:54:28 GMT]]>Also happening for me -- Falkenstein (FSN1).

]]>https://forum.cloudron.io/post/98001https://forum.cloudron.io/post/98001Tue, 10 Dec 2024 02:54:28 GMT<![CDATA[Reply to error while backing up on Mon, 09 Dec 2024 12:01:09 GMT]]>Which region of Hetzner is this? I just tested on HEL1 and multi-part copies is fine.

]]>https://forum.cloudron.io/post/97981https://forum.cloudron.io/post/97981Mon, 09 Dec 2024 12:01:09 GMT<![CDATA[Reply to error while backing up on Mon, 09 Dec 2024 10:34:23 GMT]]>Is this happening all the time? The error status code returned from Hetzner is 504 indicating an issue on the objects storage side. Cloudron here retries a few times but eventually gives up.

]]>https://forum.cloudron.io/post/97973https://forum.cloudron.io/post/97973Mon, 09 Dec 2024 10:34:23 GMT