Backups before retention policy not being deleted. Bug?

[shrey]

My backups are configured with DigitalOcean Spaces.
The Retention Policy being 3 months.

But, even though, the backup section in my Cloudron app shows backups only for the last 3 months, the DO Spaces bucket still has all the backups from the date since i set up the system, which was last year.

1. Is it by design or a bug?
2. Is there any existing method in Cloudron to delete the obsolete backups from the DO Spaces bucket?

[girish]

@shrey Missed this post, did you get this sorted out?

[shrey]

@girish The issue remains. Cloudron does not delete older backups from the S3 storage.

For now, i've manually deleted them.

[girish]

@shrey if it starts retaining obsolete backups again, can you please write to support@cloudron.io ? I can't reproduce this in any of our instances.

[shrey]

@girish That never stopped. I just deleted upto a certain date, the 'obsolete' backups after that date are still persisting.
Will send the details in the email as well.

[svtx]

I have the same issue. One backup that was done using .tar is not deleting now (5 day overdue on deletion) that I'm using rsync. I've tried reverting settings to but to no avail. The backup in question seems to be MASSIVE too, something's odd about this. Truth be told, I'd appreciate a simple "Delete Backup" option

[girish]

@svtx yes, we are looking into these improvements in 7.6

When investigating @shrey 's situation a bit, we found that stale/partial backups are not cleaned up properly. Will get this fixed in 7.6 as well.

[stormgrass]

I've been having the same issue with S3, switched to DO Spaces now because it's easier to monitor. I had terabytes of data in my S3 bucket and only noticed it once I got my bill. With DO Spaces it's now the same issue, there's still backups from three days in there despite a 2 day retention policy.

[shrey]

Hi @girish
By when can we expect a (stable) solution for this?

[shrey]

Hi @girish , pinging you again.

It's been more than 1.5 years. The issue is still ongoing.

[girish]

@shrey we can't reproduce the problem. So, I think you have to write to us at support@cloudron.io and we have to debug what is happening .

[nebulon]

We finally got down to this issue. Problem was that particular instance had a lot of backups in the db and we only took the first 1k backups into account for purging. Fix is coming with next release. If you hit this issue, send us a mail to support@cloudron.io so we can apply a temporary fix.

[shrey]

@nebulon said in Backups before retention policy not being deleted. Bug?:

Fix is coming with next release. If you hit this issue, send us a mail to support@cloudron.io so we can apply a temporary fix.

I found out yesterday that the temporary fix had gotten overwritten by the next release (v8.3.2). And i found it now because i just got hit with a nice bill for S3 overage of 15TB of backups data (just because the obsolete backups didn't get deleted as per the retention policy).

Additionally, even after reapplying the fix, the Cleanup Backups tasks are now failing, every time.

[joseph]

@shrey said in Backups before retention policy not being deleted. Bug?:

Additionally, even after reapplying the fix, the Cleanup Backups tasks are now failing, every time.

is there any more info on this? I guess the patch only patched the querying of what to clean up but didn't fix anything related to clean up logic itself .

[shrey]

@joseph
As per this advise received by support:
you could try to set that 1000 in /home/yellowtent/box/src/backupcleaner.js on your server to something like 20000 for a start. Then run "systemctl restart box" and trigger the backup cleaner task.
i had changed the 1000 into a higher value, 100000, which worked at the time, until the next Cloudron release, v8.3.2, overwrote it back to 1000.

[james]

Hello @shrey
This is expected.
Each Cloudron release and update has to treat every system the same and can not respect changes made by the user.
Changes made to the core services need to be managed by the user manually.