Backups still not verified or what?
-
So to clarify a bit further, there are two related things mixed I feel. So the latest version does calculate the hashes of the files it expects to be on the backup storage. What is not yet added is the part where the user gets notified or warned if the file on the remote storage does not pass the validation. This will come though.
The second is the actual restore issue you are facing and not really related as such to a verification issue, but the situation as such is why we introduced it, it is just not all finished (missing UI mostly)
-
@girish -> The question is: How many people using CIFS actually try to use backups? And how many are complaining? I'll write an email, just one quick question -> Would it be possible to have an option to read back the file after backup to compare it with the hash then? How else would one detect a backup problem at scale? Cause my understanding is, right now the hash does nothing, except show wether a backup is broken when I need it (which is too late to do anything about it, causing potential data loss)
@philkunz hetzner storage box (CIFS) is widely used. I don't have numbers since we don't collect them. I don't have much idea about Synology/CIFS .
Would it be possible to have an option to read back the file after backup to compare it with the hash then?
yes, that's exactly the eventual idea. I think maybe we misled you with the "What's coming in 9.0" post. I make those posts much in advance to collect feedback etc (that post was made a year ago). It doesn't mean everything there got implemented. I usually go back and edit them to indicate what got implemented and what got moved etc. I guess it's a balance between whether to communicate what's coming etc or not. We keep the development open so such inconsistencies are unavoidable.
Specifically for integrity, we are still working on this - some technical notes here and here . It's still a WIP and not exposed to user until we are sure that the integrity is reliable. It would be even worse if we assured the user that things are good when they aren't .
-
Interesting part is -> I tested with another cloudron instance that runs on another company account, but uses the same Synology backup target -> that one works. Really strange. Yet there is another user in the linked thread that has the same problem...
-
@philkunz said in Backups still not verified or what?:
Has to be a cifs problem related to large backups. NFS works, Otherwise same setup.
that is valuable information. @luckow has a synology and volunteered to help us testing this . Will wait to check if he can reproduce this. He already tested with small backups and that worked .
@luckow we need to test with large backup sizes > 20GB .
-
@philkunz said in Backups still not verified or what?:
But that points to how important the e2e read back step is.
absolutely, that is why we are adding the validation in the first place. Having checksums of what we expect is the first step, next then will be the task (can be quite heavy task, so this won't be automatic probably) to read back and recalculate the hashes.
