Backup retention time

[d19dotca]

I'm using the filesystem backup option and with a retention of 7 days. Yet I have backups still existing from a month ago. Any ideas if this is a defect? I assume retention time means any backups older than the date will be removed, but that doesn't appear to be working or how it's meant to work, not sure which yet. But perhaps I've misunderstood how that function works?

Would love some help so I can figure out how to keep the backups to 7 days if the app isn't designed to do it itself.

[mehdi]

@d19dotca Are you using RSYNC backups ? If so, maybe it's just keeping what it needs from the older backups ? Not sure though, this is just a guess

[d19dotca]

@mehdi Funny enough it is, but I'm unsure how retention works if it's mean to be different with rsync.

[mehdi]

I'm not sure. I actually also would like to know.

@girish sorry to bother you, but if you have a minute, could you try explaining the basics of how Cloudron's backup retention works with rsync ?

[girish]

There's no difference between the backup formats (the backup format is just a "format" for storing backups and doesn't affect what is actually getting stored or retained).

@d19dotca I think the behavior you are seeing could be related to backups that are created before an app update. These backups are marked as special are retained for a hardcoded 3 weeks. The reason is that you might notice only much later that something is wrong i.e the app itself works but after a couple of weeks you suddenly notice something is not right.

3 weeks is just an arbitrary time. Did you mean "a month ago" figuratively or was it really a month ago? Does the version of the backup change between that backup and the next one? That's one way to identify if it's this special update-backup.

[d19dotca]

@girish Ah, that's interesting. That isn't documented anywhere that I could find, so that wasn't known. Yes, 3 weeks sounds about right - the "a month" was more figurative as it was close. Unfortunately I can't say for certain now as the backups were failing due to disk space so I had to remove the old backups - this is how I came to learn of this discrepancy between my understanding of how retention time works and what was happening in my system.

Is this something we can customize by any chance? It's a great idea, I like that you're doing it for keeping backups when a package version change happens, because you're right - people usually don't see any smaller issues until a while later. I just wonder if it could be an option to overwrite it all for people who may not have a lot of disk space to spare. May be useful to expose that in the GUI.

On a related side note: One thing I've come to learn through using Cloudron the past 8 months or so is that there is a lot of stuff missing from the documentation that we only find out about later in the forum. I actually am the rare oddball who likes to write up documentation to help others (this is part of my day job), so I'm happy to help contribute these types of updates if you have a list or backlog of items that need to make it into the documentation. If there's anything I can offer on that side of things, I'd love to help so it improves the docs for everyone and hopefully prevents these types of issues from consuming your time.

[girish]

@d19dotca said in Backup retention time:

May be useful to expose that in the GUI.

Yes, I think a delete backup function would be useful in the UI. And maybe a hint as to why it's being kept.

so I'm happy to help contribute these types of updates if you have a list or backlog of items

That's music to my ears Our doc pages are at https://git.cloudron.io/cloudron/docs . We will put up some instruction on how to contribute to existing docs (I am not 100% sure how you can preview them correctly without the gulpfile which is in another repo). I have also recently started a guides section (so you can also write completely new articles which I can put up there).

In the initial days, the doc pages were more like a instructions manual (like how to use an oven). It didn't go deep into most topics because we didn't think it would be interesting. I think last year or so, it's clear that most people who use us are fairly technical users/sysadmins and they want to have a detailed understanding of the product. We made an internal policy that whenever we answer a question on support@ or this forum, we will edit/add information in the docs. So, with enough questions here, we will improve the docs pages. One other thing is that the search is not at all satisfactory, we are still looking for the right tool for this (something selfhosted). That's our excuse for missing info in docs

[girish]

https://cloudron.io/documentation/backups/#schedule-retention has this info now

[mehdi]

@girish said in Backup retention time:

One other thing is that the search is not at all satisfactory, we are still looking for the right tool for this (something selfhosted).

I heard good things about https://www.meilisearch.com/

Have not tried it myself though

[Mallewax]

@mehdi Is this something that could be used for Mail Full Text Search, too? I love Cloudron and feel, that the Mail Server functionality is entirely under-rated. Such a simple and slick interface. Just the one thing missing is "Full Text Search" to really come off Gmail entirely....my two cents.
Hope some gifted hands and minds can look at https://www.meilisearch.com/ more closely.
Kind of excited to see this....

[mehdi]

@Mallewax I think this depends on the webmail app, more than on the platform itself. Not totally sure though, I am not particularly familiar with the inner workings of the email protocols