After auto update to 9.0.14: backup errors

[imc67]

Last night one of my 3 Cloudron servers auto-updated to 9.0.14.

The first auto-backup after it errored (see below).
I tried to edit/safe config and remount and manual started a backup an again the same error!

Server info:

Cloudron version 9.0.14
Ubuntu version Ubuntu 22.04.5 LTS Linux 5.15.0-163-generic
Vendor netcup
Product KVM Server
CPU 10 Core "AMD EPYC 7702P 64-Core Processor"
Memory 67.42 GB RAM & 4.29 GB Swap
Uptime 4 days
Cloudron creation time unknown

Backup type:

DefaultEverything
Storage: sshfs (rsync) at u*******.your-storagebox.de:/home
Schedule: Every day @ 00:00, 08:00, 12:00, 16:00, 20:00
Retention Policy: 7 daily, 4 weekly, 12 monthly
Last run: 12 dec 2025, 08:01
SSH connection error: ssh exited with code 255 signal null 

Error:

Dec 12 08:01:53 box:shell filesystem: ssh -o "StrictHostKeyChecking no" -i /tmp/identity_file-mnt-managedbackups-6de7dc51-23ad-45e7-b6e4-137d769995ab -p 23 *****@******.your-storagebox.de rm -rf snapshot/app_1ceb591b-97ad-4793-8942-cf5853d739fb/data/public/wp-content/cache/****** errored BoxError: ssh exited with code 255 signal null
Dec 12 08:01:53 at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:82:23)
Dec 12 08:01:53 at ChildProcess.emit (node:events:519:28)
Dec 12 08:01:53 at maybeClose (node:internal/child_process:1101:16)
Dec 12 08:01:53 at ChildProcess._handle.onexit (node:internal/child_process:304:5) {
Dec 12 08:01:53 reason: 'Shell Error',
Dec 12 08:01:53 details: {},
Dec 12 08:01:53 stdout: <Buffer >,
Dec 12 08:01:53 stdoutLineCount: 0,
Dec 12 08:01:53 stderr: <Buffer 57 61 72 6e 69 6e 67 3a 20 49 64 65 6e 74 69 74 79 20 66 69 6c 65 20 2f 74 6d 70 2f 69 64 65 6e 74 69 74 79 5f 66 69 6c 65 2d 6d 6e 74 2d 6d 61 6e 61 ... 244 more bytes>,
Dec 12 08:01:53 stderrLineCount: 4,
Dec 12 08:01:53 code: 255,
Dec 12 08:01:53 signal: null,
Dec 12 08:01:53 timedOut: false,
Dec 12 08:01:53 terminated: false
Dec 12 08:01:53 }
Dec 12 08:01:53 box:backupformat/rsync sync: done processing deletes. error: BoxError: SSH connection error: ssh exited with code 255 signal null at Object.removeDir (/home/yellowtent/box/src/storage/filesystem.js:286:47) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async processSyncerChange (/home/yellowtent/box/src/backupformat/rsync.js:110:13) at async /home/yellowtent/box/src/backupformat/rsync.js:128:92 { reason: 'External Error', details: {} }
Dec 12 08:01:53 box:backupupload upload completed. error: BoxError: SSH connection error: ssh exited with code 255 signal null at Object.removeDir (/home/yellowtent/box/src/storage/filesystem.js:286:47) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async processSyncerChange (/home/yellowtent/box/src/backupformat/rsync.js:110:13) at async /home/yellowtent/box/src/backupformat/rsync.js:128:92 { reason: 'External Error', details: {} }
Dec 12 08:01:53 box:backuptask runBackupUpload: result - {"errorMessage":"SSH connection error: ssh exited with code 255 signal null"}
Dec 12 08:01:54 box:shell backuptask: /usr/bin/sudo --non-interactive -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_1ceb591b-97ad-4793-8942-cf5853d739fb 6de7dc51-23ad-45e7-b6e4-137d769995ab {"localRoot":"/home/yellowtent/appsdata/1ceb591b-97ad-4793-8942-cf5853d739fb","layout":[]} errored BoxError: /usr/bin/sudo exited with code 50 signal null
Dec 12 08:01:54 at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:82:23)
Dec 12 08:01:54 at ChildProcess.emit (node:events:519:28)
Dec 12 08:01:54 at maybeClose (node:internal/child_process:1101:16)
Dec 12 08:01:54 at ChildProcess._handle.onexit (node:internal/child_process:304:5) {
Dec 12 08:01:54 reason: 'Shell Error',
Dec 12 08:01:54 details: {},
Dec 12 08:01:54 stdout: '',
Dec 12 08:01:54 stdoutLineCount: 0,
Dec 12 08:01:54 stderr: '',
Dec 12 08:01:54 stderrLineCount: 0,
Dec 12 08:01:54 code: 50,
Dec 12 08:01:54 signal: null,
Dec 12 08:01:54 timedOut: false,
Dec 12 08:01:54 terminated: false
Dec 12 08:01:54 }
Dec 12 08:01:54 box:backuptask fullBackup: app *******.nl backup finished. Took 4.634 seconds
Dec 12 08:01:54 box:locks write: current locks: {"full_backup_task_6de7dc51-23ad-45e7-b6e4-137d769995ab":null}
Dec 12 08:01:54 box:locks release: app_backup_1ceb591b-97ad-4793-8942-cf5853d739fb
Dec 12 08:01:54 box:tasks setCompleted - 18576: {"result":null,"error":{"message":"SSH connection error: ssh exited with code 255 signal null","reason":"External Error"},"percent":100}
Dec 12 08:01:54 box:tasks updating task 18576 with: {"completed":true,"result":null,"error":{"message":"SSH connection error: ssh exited with code 255 signal null","reason":"External Error"},"percent":100}
Dec 12 08:01:54 box:taskworker Task took 24.755 seconds
Dec 12 08:01:54 BoxError: SSH connection error: ssh exited with code 255 signal null
Dec 12 08:01:54 at runBackupUpload (/home/yellowtent/box/src/backuptask.js:204:15)
Dec 12 08:01:54 at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
Dec 12 08:01:54 at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:370:34)
Dec 12 08:01:54 at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:393:26)
Dec 12 08:01:54 Exiting with code 0

[archos]

Hi, our server was updated to version 9.0.14 on Friday, and yesterday the backup failed with this error:

box:storage/filesystem SSH remote rm failed, trying sshfs rm
Dec 15 05:06:26 box:shell filesystem: rm -rf /mnt/managedbackups/0a6effb5-6b5d-48f6-9e8d-2a0bd2f3e610/oscloud/snapshot/app_283b2bb3-155e-4d7b-9f9b-410e24888efb/data/data/tymoty/files_trashbin/versions/Shared (zkopírovat).d1763126576
Dec 15 05:06:27 box:shell filesystem: ssh -o "StrictHostKeyChecking no" -i /tmp/identity_file-mnt-managedbackups-0a6effb5-6b5d-48f6-9e8d-2a0bd2f3e610 -p 23 u430632@u430632.your-storagebox.de rm -rf oscloud/snapshot/app_283b2bb3-155e-4d7b-9f9b-410e24888efb/data/data/prvosienka/files_encryption/keys/files_trashbin/files/25-11-10 10-52-51 4362.jpg.d1763121179 errored BoxError: ssh exited with code 255 signal null
Dec 15 05:06:27 at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:82:23)
Dec 15 05:06:27 at ChildProcess.emit (node:events:519:28)
Dec 15 05:06:27 at maybeClose (node:internal/child_process:1101:16)
Dec 15 05:06:27 at Socket.<anonymous> (node:internal/child_process:456:11)
Dec 15 05:06:27 at Socket.emit (node:events:519:28)
Dec 15 05:06:27 at Pipe.<anonymous> (node:net:346:12) {
Dec 15 05:06:27 reason: 'Shell Error',
Dec 15 05:06:27 details: {},
Dec 15 05:06:27 stdout: <Buffer >,
Dec 15 05:06:27 stdoutLineCount: 0,
Dec 15 05:06:27 stderr: <Buffer 6e 6f 20 73 75 63 68 20 69 64 65 6e 74 69 74 79 3a 20 2f 74 6d 70 2f 69 64 65 6e 74 69 74 79 5f 66 69 6c 65 2d 6d 6e 74 2d 6d 61 6e 61 67 65 64 62 61 ... 224 more bytes>,
Dec 15 05:06:27 stderrLineCount: 4,
Dec 15 05:06:27 code: 255,
Dec 15 05:06:27 signal: null,
Dec 15 05:06:27 timedOut: false,
Dec 15 05:06:27 terminated: false
Dec 15 05:06:27 }
Dec 15 05:06:27 box:backupformat/rsync sync: done processing deletes. error: BoxError: SSH connection error: ssh exited with code 255 signal null at Object.removeDir (/home/yellowtent/box/src/storage/filesystem.js:286:47) at async processSyncerChange (/home/yellowtent/box/src/backupformat/rsync.js:110:13) at async /home/yellowtent/box/src/backupformat/rsync.js:128:92 { reason: 'External Error', details: {} }
Dec 15 05:06:27 box:backupupload upload completed. error: BoxError: SSH connection error: ssh exited with code 255 signal null at Object.removeDir (/home/yellowtent/box/src/storage/filesystem.js:286:47) at async processSyncerChange (/home/yellowtent/box/src/backupformat/rsync.js:110:13) at async /home/yellowtent/box/src/backupformat/rsync.js:128:92 { reason: 'External Error', details: {} }
Dec 15 05:06:27 box:backuptask runBackupUpload: result - {"errorMessage":"SSH connection error: ssh exited with code 255 signal null"}
Dec 15 05:06:27 box:shell backuptask: /usr/bin/sudo --non-interactive -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_283b2bb3-155e-4d7b-9f9b-410e24888efb 0a6effb5-6b5d-48f6-9e8d-2a0bd2f3e610 {"localRoot":"/home/yellowtent/appsdata/283b2bb3-155e-4d7b-9f9b-410e24888efb","layout":[]} errored BoxError: /usr/bin/sudo exited with code 50 signal null
Dec 15 05:06:27 at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:82:23)
Dec 15 05:06:27 at ChildProcess.emit (node:events:519:28)
Dec 15 05:06:27 at maybeClose (node:internal/child_process:1101:16)
Dec 15 05:06:27 at Socket.<anonymous> (node:internal/child_process:456:11)
Dec 15 05:06:27 at Socket.emit (node:events:519:28)
Dec 15 05:06:27 at Pipe.<anonymous> (node:net:346:12) {
Dec 15 05:06:27 reason: 'Shell Error',
Dec 15 05:06:27 details: {},
Dec 15 05:06:27 stdout: '',
Dec 15 05:06:27 stdoutLineCount: 0,
Dec 15 05:06:27 stderr: '',
Dec 15 05:06:27 stderrLineCount: 0,
Dec 15 05:06:27 code: 50,
Dec 15 05:06:27 signal: null,
Dec 15 05:06:27 timedOut: false,
Dec 15 05:06:27 terminated: false
Dec 15 05:06:27 }
Dec 15 05:06:27 box:backuptask fullBackup: app next.oscloud.cz backup finished. Took 51.625 seconds
Dec 15 05:06:27 box:locks write: current locks: {"full_backup_task_0a6effb5-6b5d-48f6-9e8d-2a0bd2f3e610":null}
Dec 15 05:06:27 box:locks release: app_backup_283b2bb3-155e-4d7b-9f9b-410e24888efb
Dec 15 05:06:27 box:tasks setCompleted - 9744: {"result":null,"error":{"message":"SSH connection error: ssh exited with code 255 signal null","reason":"External Error"},"percent":100}
Dec 15 05:06:27 box:tasks updating task 9744 with: {"completed":true,"result":null,"error":{"message":"SSH connection error: ssh exited with code 255 signal null","reason":"External Error"},"percent":100}
Dec 15 05:06:27 box:taskworker Task took 127.881 seconds
Dec 15 05:06:27 BoxError: SSH connection error: ssh exited with code 255 signal null
Dec 15 05:06:27 at runBackupUpload (/home/yellowtent/box/src/backuptask.js:204:15)
Dec 15 05:06:27 at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:370:34)
Dec 15 05:06:27 at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:393:26)
Dec 15 05:06:27 Exiting with code 0

Quick datapoint: our backup to Hetzner Storage Box (SSH port 23) fails in the delete phase with no such identity: /tmp/identity_file-mnt-managedbackups-... and ssh exited with code 255. Started after upgrade to 9.0.14.

[robi]

@archos you try the fix in 9.0.15?