Backup failed on backblaze. Status 408?

[daixiwen]

Hello,

I've had several times now a failure when doing a backup. It says "Backup failed: Backuptask crashed. Logs are available". Here is the log:

[no timestamp]  /home/yellowtent/box/node_modules/aws-sdk/lib/services/s3.js:712
[no timestamp]  resp.error = AWS.util.error(new Error(), {
[no timestamp]  ^
[no timestamp] 
[no timestamp]  null
[no timestamp]  at Request.extractError (/home/yellowtent/box/node_modules/aws-sdk/lib/services/s3.js:712:35)
[no timestamp]  at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
[no timestamp]  at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
[no timestamp]  at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)
[no timestamp]  at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
[no timestamp]  at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
[no timestamp]  at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
[no timestamp]  at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)
[no timestamp]  at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:688:12)
[no timestamp]  at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
[no timestamp]  code: 408,
[no timestamp]  region: null,
[no timestamp]  time: 2024-10-01T00:03:35.783Z,
[no timestamp]  requestId: null,
[no timestamp]  extendedRequestId: undefined,
[no timestamp]  cfId: undefined,
[no timestamp]  statusCode: 408,
[no timestamp]  retryable: false,
[no timestamp]  retryDelay: 20000
[no timestamp]  }
[no timestamp] 
[no timestamp]  v20.12.2
Oct 01 02:03:35 box:shell backup-snapshot/app_31cc776c-dfdd-4708-a74e-40df53fa1b2a: /usr/bin/sudo -S -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_31cc776c-dfdd-4708-a74e-40df53fa1b2a tgz {"localRoot":"/home/yellowtent/appsdata/31cc776c-dfdd-4708-a74e-40df53fa1b2a","layout":[]} errored BoxError: backup-snapshot/app_31cc776c-dfdd-4708-a74e-40df53fa1b2a exited with code 1 signal null
[no timestamp]  at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:122:19)
[no timestamp]  at ChildProcess.emit (node:events:518:28)
[no timestamp]  at ChildProcess.emit (node:domain:488:12)
[no timestamp]  at ChildProcess._handle.onexit (node:internal/child_process:294:12) {
[no timestamp]  reason: 'Shell Error',
[no timestamp]  details: {},
[no timestamp]  code: 1,
[no timestamp]  signal: null
[no timestamp]  }
Oct 01 02:03:35 box:taskworker Task took 211.14 seconds
Oct 01 02:03:35 box:tasks setCompleted - 2407: {"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)\n at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}}
Oct 01 02:03:35 box:tasks update 2407: {"percent":100,"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)\n at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}}
[no timestamp]  Backuptask crashed
[no timestamp]  at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)
[no timestamp]  at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[no timestamp]  at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)
[no timestamp]  at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)
[no timestamp]  at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)

It looks like the provider (Backblaze) is returning a status 408, but the script doesn't know how to handle it, or am I reading this wrong?
According to [url=https://www.backblaze.com/docs/cloud-storage-native-api-error-handling-and-status-codes]Backblaze's documentation[/url] a status 408 is a timeout.

Is there anything I can do, other than changing my backup provider?

[joseph]

@daixiwen Can you quickly check if Backups -> Configure -> Save without making any changes gives an error ? This will do a test upload to Backblaze.

[daixiwen]

I don't get any error. I must add that usually the backup works (give or take a few internal errors from BackBlaze). The "backup task crashed" only happened twice. The first time was on september 10th and the other one was yesterday.

[joseph]

@daixiwen ah ok. I would ignore the errors. It's normal for upstream services to go down once in a while...

I just checked https://status.backblaze.com/ and unfortunately no history of uptime here. But it says "Backblaze performs regular maintenance every Thursday from 11:30 am to 1:30 p.m. Pacific Time with the aim to consistently improve our systems and services. "