Solved Cloudron not removing expired backups from deleted apps

[d19dotca]

I'm not sure if this is intentional or not, but it seems "wrong" to me so wanted to raise this. I noticed in reviewing my backups recently that it still had many from way past the retention period and the commonality in them all were they were apps that had been removed (i.e. running for a week or so to test things then deleted from Cloudron), so it seems like Cloudron is not deleting backups after the retention policy expires for previously deleted applications.

[girish]

@d19dotca Thanks for reporting. This seems like a bug/regression because all app backups are cleaned up per backup policy and the code doesn't depend on installation state.

Let me test this and get back.

[girish]

@d19dotca I tested this out and it seems they are getting removed for me. If you run the 'Cleanup backups' manually, can you check the logs why those specific backups are getting retained ?

Note that if an app was updated and then later deleted, the quirk of the backup system is that the backups that got created during the app backup are still retained for 3 weeks. Could this be the case?

[d19dotca]

@girish said in Cloudron not removing expired backups from deleted apps:

Note that if an app was updated and then later deleted, the quirk of the backup system is that the backups that got created during the app backup are still retained for 3 weeks. Could this be the case?

I don't think it'd have been that because the backups went as far back as November 2022, well over the 3 weeks.

@girish said in Cloudron not removing expired backups from deleted apps:

If you run the 'Cleanup backups' manually, can you check the logs why those specific backups are getting retained ?

This may be difficult as I had run through and manually removed them when I reported the issue (sorry about that). I did it now though and don't see ay errors, but may be because I already removed a lot of the outdated ones earlier.

[girish]

Alright, so if someone else hits this please leave a note here.

[d19dotca]

@girish Somewhat related, I decided to clear out all my 2022 backups, and I then ran the Cleanup Backups option in Cloudron to keep things in sync. Unfortunately it still shows a backup from November 30th 2022 in the list of Box Backups. I got the cleanup logs from before the deletion of the 2022 backups and then the cleanup logs from after the deletion of 2022 backups. There seems to be something broken at least in my environment with regards to the cleanup functionality in Cloudron backups.

Currently I'm not seeing any real differences when comparing them. Here's the end of the logs for the latest run, doesn't seem like anything was removed but not sure if I'm reading it correctly:

2023-02-09T05:59:29.699Z box:backupcleaner cleanupAppBackups: done
2023-02-09T05:59:29.699Z box:tasks update 19006: {"percent":70,"message":"Checking storage backend and removing stale entries in database"}
2023-02-09T05:59:41.204Z box:backupcleaner cleanupMissingBackups: done
2023-02-09T05:59:41.204Z box:tasks update 19006: {"percent":90,"message":"Cleaning snapshots"}
2023-02-09T05:59:41.316Z box:backupcleaner cleanupSnapshots: done
2023-02-09T05:59:41.316Z box:taskworker Task took 11.845 seconds
2023-02-09T05:59:41.316Z box:tasks setCompleted - 19006: {"result":{"removedBoxBackupPaths":[],"removedMailBackupPaths":[],"removedAppBackupPaths":[],"missingBackupPaths":[]},"error":null}
2023-02-09T05:59:41.316Z box:tasks update 19006: {"percent":100,"result":{"removedBoxBackupPaths":[],"removedMailBackupPaths":[],"removedAppBackupPaths":[],"missingBackupPaths":[]},"error":null}

[girish]

@d19dotca Did you change the backup provider in the past? Currently, if you switch things, we lose track of the backups on the remote (Even if you switch it back)

[d19dotca]

@girish I’ve changed backup providers before but not since October sometime. It’s been the same for a few months now. But even if I had changed backup providers, isn’t the cleanup supposed to remove any backups from Cloudron that it doesn’t see in the current setup? Maybe I’ve misunderstood that part.

[girish]

@d19dotca said in Cloudron not removing expired backups from deleted apps:

But even if I had changed backup providers, isn’t the cleanup supposed to remove any backups from Cloudron that it doesn’t see in the current setup?

What I meant is:

1. You had backups in service A
2. You changed backups to service B
3. You changed backups back to service A

In the above situation, Cloudron loses track of backups in service A after step 2. When the backup cleaner runs, it will see that backup entries in it's database don't exist in the remote (now B) anymore and it will remove them from it's database. When you then do step 3, it won't delete those old backups because it has lost track of them.

This may not apply to you, but just something to keep in mind.

[d19dotca]

@girish said in Cloudron not removing expired backups from deleted apps:

@d19dotca said in Cloudron not removing expired backups from deleted apps:

But even if I had changed backup providers, isn’t the cleanup supposed to remove any backups from Cloudron that it doesn’t see in the current setup?

What I meant is:

1. You had backups in service A
2. You changed backups to service B
3. You changed backups back to service A

In the above situation, Cloudron loses track of backups in service A after step 2. When the backup cleaner runs, it will see that backup entries in it's database don't exist in the remote (now B) anymore and it will remove them from it's database. When you then do step 3, it won't delete those old backups because it has lost track of them.

This may not apply to you, but just something to keep in mind.

Oh I see what you mean now. Okay yeah anytime I've changed providers I've also done that "reset" thing you taught me by changing Backups to no-op, running the Clean Backups part, and then back again to the new provider settings to avoid all those linking issues.