Backup Fails NEXTCLOUD

[savity]

Hi failing backups since 7 Days in a row

it only not work with Nextcloud 4 other apps do work

Cloudron 6.0.1

But i don't see any errors in the Backup Logs?

BR
Savity

[subven]

Can you post your (censored) backup configuration? Is there enough space at your backup destination for Nextcloud to be backed up?

[girish]

@savity said in Backup Fails NEXTCLOUD:

But i don't see any errors in the Backup Logs?

So what is in the logs? Can you paste the last 100 lines or so to https://paste.cloudron.io ?

[savity]

@girish there was really no error but since theni raised the backup RAM and it worked for 1 Week.

Now i got a new error

Jan 19 02:11:14 box:tasks 2859: {"percent":51,"message":"Uploading backup 4473M@7MBps (SERVER.domain.domain)"}
Jan 19 02:11:20 box:shell backup-snapshot/APP_ID (stdout): 2021-01-19T01:11:20.928Z box:storage/s3 Error uploading [SERVER/snapshot/APP_ID.tar.gz.enc]: s3 upload error. { InternalError: We encountered an internal error, please try again.
at Request.extractError (/home/yellowtent/box/node_modules/aws-sdk/lib/services/s3.js:837:35)
at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:688:14)
at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:690:12)
at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
message: 'We encountered an internal error, please try again.',
code: 'InternalError',
region: null,
time: 2021-01-19T01:11:20.924Z,
requestId: '165B7CF26E4F51D8',
extendedRequestId: undefined,
cfId: undefined,
statusCode: 500,
retryable: true }
Jan 19 02:11:20 box:shell backup-snapshot/APP_ID (stdout): 2021-01-19T01:11:20.929Z box:backupupload upload completed. error: { BoxError: Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError
at ManagedUpload.callback (/home/yellowtent/box/src/storage/s3.js:130:33)
at ManagedUpload.cleanup (/home/yellowtent/box/node_modules/aws-sdk/lib/s3/managed_upload.js:629:10)
at Response.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/s3/managed_upload.js:566:28)
at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:369:18)
at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:688:14)
at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
name: 'BoxError',
reason: 'External Error',
details: {},
message:
'Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError' }
Jan 19 02:11:20 box:backups runBackupUpload: result - {"result":"Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError"}
Jan 19 02:11:21 box:shell backup-snapshot/APP_ID code: 50, signal: null
Jan 19 02:11:21 box:backups SERVER.domain.domain Unable to backup { BoxError: Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError
at /home/yellowtent/box/src/backups.js:874:29
at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)
at ChildProcess.emit (events.js:198:13)
at ChildProcess.EventEmitter.emit (domain.js:448:20)
at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
name: 'BoxError',
reason: 'External Error',
details: {},
message:
'Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError' }
Jan 19 02:11:21 box:taskworker Task took 680.211 seconds
Jan 19 02:11:21 box:tasks setCompleted - 2859: {"result":null,"error":{"stack":"BoxError: Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError\n at /home/yellowtent/box/src/backups.js:874:29\n at f (/home/yellowtent/box/node_modules/once/once.js:25:25)\n at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)\n at ChildProcess.emit (events.js:198:13)\n at ChildProcess.EventEmitter.emit (domain.js:448:20)\n at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)","name":"BoxError","reason":"External Error","details":{},"message":"Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError"}}
Jan 19 02:11:21 box:tasks 2859: {"percent":100,"result":null,"error":{"stack":"BoxError: Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError\n at /home/yellowtent/box/src/backups.js:874:29\n at f (/home/yellowtent/box/node_modules/once/once.js:25:25)\n at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)\n at ChildProcess.emit (events.js:198:13)\n at ChildProcess.EventEmitter.emit (domain.js:448:20)\n at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)","name":"BoxError","reason":"External Error","details":{},"message":"Error uploading SERVER/snapshot/APP_ID.tar.gz.enc. Message: We encountered an internal error, please try again. HTTP Code: InternalError"}}

[savity]

@girish and some time a go you have given me a soluton:

@savity Can you try this change https://git.cloudron.io/cloudron/box/-/commit/bedcd6fccf58830b316318699375bc1f582a5d7a ? The file on Cloudron is /home/yellowtent/box/src/storage/s3.js. Essentially change the timeout from 3000 * 1000 to 0. And also change maxRetries to 10. You don't need to restart anything after the change since the code changes are immediately picked up.

I am using MINIO as Backup and i think this all started after the last Ubuntu security update

[girish]

@savity The error message "InternalError: We encountered an internal error, please try again" indicates this was an error from S3. So maybe this was just a sporadic failure. Do you consistently get the error?

[savity]

@girish It didn't worked for 4 Days now in a row