Backups Listing not showing recent backups

[d19dotca]

I noticed suddenly today that my Backups Listings aren't showing correctly... it's missing the last few backups (there are three a day, with a 2 day 4 week retention policy). Oddly, I can see the backups are fine each app, so it seems like something is just not being logged properly in the Listing part on the main Backups page.

What the current Listings looks like (the one at "40 minutes ago" was the one I did manually as I was concerned by the lack of anything showing for a week on this page):

What it looks like as expected though on an app's backup listings:

Why was yesterday's backup for example not showing on the main Backups Listings page? It's concerning, because while I believe the backup exists, it isn't shown there which makes it a little difficult to download the backup details if I ever had to restore quickly, for example.

[d19dotca]

@girish Ah I didn't see this until too late, sorry. I had tried to start over to a degree by destroying the current backup disk and mounting a new disk instead then clearing out the backups history and seeing if that starts working better for me. So far so good, but if the issue comes back I'll certainly send the email out and enable remote support, as I agree there seems like some odd issue for sure.

[girish]

@d19dotca a backup appears on app listing page if it was a partial backup. so matomo worked but it failed elsewhere. Can you check the logs of the automatic backups and see where they failed?

You can do this by Event log -> Backup.start event and then note the taskId. From there, check /home/yellowtent/platformdata/logs/tasks/xxx.log

(as a heads up, there is a bug in the latest release that events are not listed properly in the UI. So, don't worry if events seem missing).

[d19dotca]

@girish I’ll check the logs when I’m back at my desktop, right now I’m just in my phone. I logged in and only see the latest backup from 2 weeks ago, the one from a couple days ago seems gone from the main backups page. If backups were failing for some reason, shouldn’t it also fail to remove backups in such a scenario as to avoid losing the most recent backup? Maybe that’s a side issue but the question popped into my head.

[d19dotca]

@girish I'm not seeing any errors in the backup task logs... it seems to complete without errors.

Jun 21 21:17:47 box:shell backup-snapshot/mail (stdout): 2022-06-22T04:17:47.586Z box:syncer Done processing adds. error: null
Jun 21 21:17:47 box:shell backup-snapshot/mail (stdout): 2022-06-22T04:17:47.588Z box:backupupload upload completed. error: null
Jun 21 21:17:47 box:backuptask runBackupUpload: result - {"result":""}
Jun 21 21:17:47 box:backuptask uploadMailSnapshot: took 638.113 seconds
Jun 21 21:17:47 box:backuptask rotateMailBackup: rotating to 2022-06-22-040001-379/mail_v7.2.4
Jun 21 21:17:47 box:tasks update 15679: {"percent":96.2380952380952,"message":"Copying /mnt/cloudron-backups-2/snapshot/mail to /mnt/cloudron-backups-2/2022-06-22-040001-379/mail_v7.2.4"}
Jun 21 21:17:47 box:shell copy spawn: /bin/cp -al /mnt/cloudron-backups-2/snapshot/mail /mnt/cloudron-backups-2/2022-06-22-040001-379/mail_v7.2.4
Jun 21 21:17:55 box:backuptask copy: copied successfully to 2022-06-22-040001-379/mail_v7.2.4. Took 7.689 seconds
Jun 21 21:17:55 box:tasks update 15679: {"percent":96.2380952380952,"message":"Backing up system data"}
Jun 21 21:17:55 box:tasks update 15679: {"percent":98.61904761904758,"message":"Snapshotting box"}
Jun 21 21:17:55 box:shell exportToFile exec: /usr/bin/mysqldump -h "127.0.0.1" -u root -ppassword --column-statistics=0 --single-transaction --routines --triggers box > "/home/yellowtent/boxdata/box/box.mysqldump"
Jun 21 21:17:56 box:shell exportToFile (stdout): null
Jun 21 21:17:56 box:shell exportToFile (stderr): mysqldump: [Warning] Using a password on the command line interface can be insecure.
Jun 21 21:17:56 box:backuptask snapshotBox: took 1.116 seconds
Jun 21 21:17:56 box:tasks update 15679: {"percent":98.61904761904758,"message":"Uploading box snapshot"}
Jun 21 21:17:56 box:shell backup-snapshot/box spawn: /usr/bin/sudo -S -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/box rsync {"localRoot":"/home/yellowtent/boxdata/box","layout":[]}
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.395Z box:backupupload Backing up {"localRoot":"/home/yellowtent/boxdata/box","layout":[]} to snapshot/box
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.445Z box:settings initCache: pre-load settings
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.462Z box:backupupload process: rss: 69.92MB heapUsed: 24.1MB heapTotal: 35.78MB external: 6.72MB
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.462Z box:backupupload v8 heap: used 24.1MB total: 35.78MB max: 2.05GB
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.463Z box:backuptask upload: path snapshot/box format rsync dataLayout {"localRoot":"/home/yellowtent/boxdata/box","layout":[]}
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.486Z box:storage/filesystem checkPreconditions: getting disk usage of /home/yellowtent/boxdata/box
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.492Z box:storage/filesystem checkPreconditions: 23876662 bytes
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.531Z box:syncer Processing 0 deletes and 2 additions
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.531Z box:syncer Done processing deletes. error: null
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.532Z box:backupformat/rsync sync: processing task: {"operation":"add","path":"box.mysqldump","reason":"changed","position":0}
Jun 21 21:17:57 box:tasks update 15679: {"percent":98.61904761904758,"message":"Adding box.mysqldump (box)"}
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.533Z box:backupformat/rsync Adding box.mysqldump position 0 try 1
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.534Z box:backupformat/rsync sync: processing task: {"operation":"add","path":"fsmetadata.json","reason":"changed","position":1}
Jun 21 21:17:57 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:57.535Z box:backupformat/rsync Adding fsmetadata.json position 1 try 1
Jun 21 21:17:58 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:58.019Z box:storage/filesystem upload /mnt/cloudron-backups-2/snapshot/box/fsmetadata.json: done.
Jun 21 21:17:58 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:58.019Z box:backupformat/rsync Uploaded fsmetadata.json
Jun 21 21:17:58 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:58.166Z box:storage/filesystem upload /mnt/cloudron-backups-2/snapshot/box/box.mysqldump: done.
Jun 21 21:17:58 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:58.166Z box:backupformat/rsync Uploaded box.mysqldump
Jun 21 21:17:58 box:syncer Done processing adds. error: null
Jun 21 21:17:58 box:shell backup-snapshot/box (stdout): 2022-06-22T04:17:58.166Z box:backupupload upload completed. error: null
Jun 21 21:17:58 box:backuptask runBackupUpload: result - {"result":""}
Jun 21 21:17:58 box:backuptask uploadBoxSnapshot: took 1.71 seconds
Jun 21 21:17:58 box:backuptask rotateBoxBackup: rotating to id 2022-06-22-040001-379/box_v7.2.4
Jun 21 21:17:58 box:tasks update 15679: {"percent":98.61904761904758,"message":"Copying /mnt/cloudron-backups-2/snapshot/box to /mnt/cloudron-backups-2/2022-06-22-040001-379/box_v7.2.4"}
Jun 21 21:17:58 box:shell copy spawn: /bin/cp -al /mnt/cloudron-backups-2/snapshot/box /mnt/cloudron-backups-2/2022-06-22-040001-379/box_v7.2.4
Jun 21 21:17:58 box:backuptask copy: copied successfully to 2022-06-22-040001-379/box_v7.2.4. Took 0.088 seconds
Jun 21 21:17:58 box:taskworker Task took 1077.006 seconds
Jun 21 21:17:58 box:tasks setCompleted - 15679: {"result":"box_box_v7.2.4_c98c845a4a4d72b10b6c0be26d1742b0470195ad9306fa62f55fa6033d9a6d3f","error":null}
Jun 21 21:17:58 box:tasks update 15679: {"percent":100,"result":"box_box_v7.2.4_c98c845a4a4d72b10b6c0be26d1742b0470195ad9306fa62f55fa6033d9a6d3f","error":null}

As far as I can tell, there are no errors at all during the process and the backups show fine on every app, it's only the main Backups page that doesn't show recent backups for some reason.

[girish]

@d19dotca do you think you can enable remote access and drop me a mail at support@cloudron.io ? Something is off, definitely.

[d19dotca]

@girish Ah I didn't see this until too late, sorry. I had tried to start over to a degree by destroying the current backup disk and mounting a new disk instead then clearing out the backups history and seeing if that starts working better for me. So far so good, but if the issue comes back I'll certainly send the email out and enable remote support, as I agree there seems like some odd issue for sure.