Backups before retention policy not being deleted. Bug?

[girish]

@svtx yes, we are looking into these improvements in 7.6

When investigating @shrey 's situation a bit, we found that stale/partial backups are not cleaned up properly. Will get this fixed in 7.6 as well.

[stormgrass]

I've been having the same issue with S3, switched to DO Spaces now because it's easier to monitor. I had terabytes of data in my S3 bucket and only noticed it once I got my bill. With DO Spaces it's now the same issue, there's still backups from three days in there despite a 2 day retention policy.

[shrey]

Hi @girish
By when can we expect a (stable) solution for this?

[shrey]

Hi @girish , pinging you again.

It's been more than 1.5 years. The issue is still ongoing.

[girish]

@shrey we can't reproduce the problem. So, I think you have to write to us at support@cloudron.io and we have to debug what is happening .

[nebulon]

We finally got down to this issue. Problem was that particular instance had a lot of backups in the db and we only took the first 1k backups into account for purging. Fix is coming with next release. If you hit this issue, send us a mail to support@cloudron.io so we can apply a temporary fix.

[shrey]

@nebulon said in Backups before retention policy not being deleted. Bug?:

Fix is coming with next release. If you hit this issue, send us a mail to support@cloudron.io so we can apply a temporary fix.

I found out yesterday that the temporary fix had gotten overwritten by the next release (v8.3.2). And i found it now because i just got hit with a nice bill for S3 overage of 15TB of backups data (just because the obsolete backups didn't get deleted as per the retention policy).

Additionally, even after reapplying the fix, the Cleanup Backups tasks are now failing, every time.

[joseph]

@shrey said in Backups before retention policy not being deleted. Bug?:

Additionally, even after reapplying the fix, the Cleanup Backups tasks are now failing, every time.

is there any more info on this? I guess the patch only patched the querying of what to clean up but didn't fix anything related to clean up logic itself .

[shrey]

@joseph
As per this advise received by support:
you could try to set that 1000 in /home/yellowtent/box/src/backupcleaner.js on your server to something like 20000 for a start. Then run "systemctl restart box" and trigger the backup cleaner task.
i had changed the 1000 into a higher value, 100000, which worked at the time, until the next Cloudron release, v8.3.2, overwrote it back to 1000.

[james]

Hello @shrey
This is expected.
Each Cloudron release and update has to treat every system the same and can not respect changes made by the user.
Changes made to the core services need to be managed by the user manually.