Ability to synchronise backup and apps update schedule

[ruihildt]

I started to think about backup schedule, because I currently use for backups Exoscale, which doesn't have a storage lifecycle methods.
Right now, every X months, I go to Exoscale buckets and manually delete all my weekly backups except one per month for the past 6 months and then keep 1 backup per year. I need to do that otherwise my backup bills go through the roof.

So I started to think about creating an app using Exoscale API to do this for me. But right now, it's a mess because there are two types of backups: the complete backups and the partial apps backup triggered by apps update.

Through the Settings, we can choose the app update schedule:

• every night
• wednesday night
• saturday night

Through the Backups configuration, we can choose the backup interval:

• every 6 hours
• every 12 hours
• every day
• every 3 days
• every week

Right now, if I select weekly (which I have), I have no idea when the cloudron backup will be done.

Apps update happens either daily or weekly.
Backups also happens at least at the same frequency.

Which means it should be possible to synchronize both operations: we could have the full backups done as part of the scheduled apps update (so before the update): it would ensure a full backup is done, and minimize the space used by backups. (and lower costs, yay!)

The only case where that would be an issue is with the 3 days backups: it doesn't fit the weekly apps update schedule. But it can be mitigated by changing from 3 days to 3.5 days, which is half a week.^^
And manual backups/apps update could still be triggered manually without interfering with the schedule.

Ok, that was quite long.
I'm would be interested to have your thoughts on this (and also discuss about alternative strategies to do lifecycle backups management).

[ruihildt]

Also, what is the best way to differentiate a full backup from a partial backup?
Is checking whether the backup contains a file starting with box_ a correct way?

[RoboMod]

I had the exact same issue. I'm using a minio installation on a second machine. The backups are created every day. And then I wrote a python scripting managing those backups which means deleting those not necessary according to the backup scheme (1 per year, 3 month, 4 weeks, and the last 7 days). This script runs on my Cloudron machine. So, backups are only deleted when Cloudron is up and running and has connection to the backup machine. And yes, full backups are the ones having a box_ directory ;).

[ruihildt]

@RoboMod Would you be willing to share your script?

[RoboMod]

Em, yes, of course. I'll try to make it available in the next days.

[RoboMod]

Here it is: https://git.cloudron.io/RoboMod/minio-auto-deleter. If there are any questions, don't hesitate to ask!

[girish]

@RoboMod Thanks for the script and @ruihildt thanks for the suggestion. We will look into this in a coming release. It makes sense to keep them in sync. I have long wanted better backup rotation policies as well - https://git.cloudron.io/cloudron/box/issues/441