Backups still not verified or what?
-
Not at all. Hashing is very well suited for data integrity. Not sure why live snapshots of file system is needed . one two three etc. Plenty of articles to read about this.
I think but for a start we need to know if what Cloudron thinks it saved on the backup disk matches what is currently on the disk. This is what computing a checksum will tell us.
-
I needed exactly 2 backups to work from 2 different apps in the last 3 month. Both did not work. The question is: What exactly are you hashing? Where are you creating the hash? How are you creating the hash? A hash is simply a function that can tell you with a certain probability, that a certain input is equal to another input if the hash matches. It tells you nothing about wether the backup works or not. It might increase your odds, but nothing more.
Just tested with another app -> also failing. And yes, there are hash mismatches. Just tested a few. Hashes do not match.
-
To put it bluntly: Working Backups are more important than a refined UI, new features, or a new app. If mismatching hashes are not even detected automatically, there is work to be done before doing anything else. Otherwise this in my view is unresponsible negligence for a product like this.
-
-
@philkunz the hash is the hash of the file Cloudron saved to the storage. If they are not matching, then it either means the initial upload by Cloudron was wrong or something is off with the CIFS storage.
I can't reproduce this with hetzner CIFS at least which is to say it's not a general problem. Can you get in touch at support@cloudron.io and we can debug this further . Thanks!
-
@girish -> The question is: How many people using CIFS actually try to use backups? And how many are complaining? I'll write an email, just one quick question -> Would it be possible to have an option to read back the file after backup to compare it with the hash then? How else would one detect a backup problem at scale? Cause my understanding is, right now the hash does nothing, except show wether a backup is broken when I need it (which is too late to do anything about it, causing potential data loss)
-
So to clarify a bit further, there are two related things mixed I feel. So the latest version does calculate the hashes of the files it expects to be on the backup storage. What is not yet added is the part where the user gets notified or warned if the file on the remote storage does not pass the validation. This will come though.
The second is the actual restore issue you are facing and not really related as such to a verification issue, but the situation as such is why we introduced it, it is just not all finished (missing UI mostly)
-
@girish -> The question is: How many people using CIFS actually try to use backups? And how many are complaining? I'll write an email, just one quick question -> Would it be possible to have an option to read back the file after backup to compare it with the hash then? How else would one detect a backup problem at scale? Cause my understanding is, right now the hash does nothing, except show wether a backup is broken when I need it (which is too late to do anything about it, causing potential data loss)
@philkunz hetzner storage box (CIFS) is widely used. I don't have numbers since we don't collect them. I don't have much idea about Synology/CIFS .
Would it be possible to have an option to read back the file after backup to compare it with the hash then?
yes, that's exactly the eventual idea. I think maybe we misled you with the "What's coming in 9.0" post. I make those posts much in advance to collect feedback etc (that post was made a year ago). It doesn't mean everything there got implemented. I usually go back and edit them to indicate what got implemented and what got moved etc. I guess it's a balance between whether to communicate what's coming etc or not. We keep the development open so such inconsistencies are unavoidable.
Specifically for integrity, we are still working on this - some technical notes here and here . It's still a WIP and not exposed to user until we are sure that the integrity is reliable. It would be even worse if we assured the user that things are good when they aren't .
-
Interesting part is -> I tested with another cloudron instance that runs on another company account, but uses the same Synology backup target -> that one works. Really strange. Yet there is another user in the linked thread that has the same problem...
