Backup Improvements: Restic Backend
-
@necrevistonnezr that's an excellent post which I forgot about! I just proceeded to set up a backup of my laptop with restic+minio. Let's see how it goes!
-
After a year, prune fails randomly
The index references 1 needed pack files which are missing from the repository: 64c2b4cd29f07b6d64f4d0ede2509fd680597b16db65b77e14e8575e1265f1bf Fatal: packs from index missing in repo -
wrote on Feb 5, 2025, 1:07 PM last edited by
https://forum.restic.net/t/fatal-packs-from-index-missing-in-repo/4869 --> https://github.com/restic/restic/issues/828#issuecomment-706186047 --> https://restic.readthedocs.io/en/stable/077_troubleshooting.html
Judging from the background story in Very slow restic prune , my guess would be that the S3 bucket listing is incomplete from time to time. So, a quite likely explanation is that the list of pack files which prune gets in incomplete and hence the missing file errors. As far as I remember, minio has options to ensure a consistent file listing or maybe you’re encountering some timeout.
-
@necrevistonnezr thanks. so far, I am following the troubleshooting guide and running check --read-data first. This runs for around 3 hours now, let's hope it says something at the end of it. We are using Vultr Storage.
-
wrote on Feb 5, 2025, 1:33 PM last edited by necrevistonnezr Feb 5, 2025, 1:35 PM
For what it's worth, I had zero problems in the last 7 years I've been using restic on my server with around 380 GB of data to be backed up to Onedrive (via rclone)...
I do regular tests with restic and spot checks via restic-browser. -
wrote on Feb 5, 2025, 2:07 PM last edited by
@necrevistonnezr how have you been testing backups? I have a raspberry pi at home, set up with restic/rclone backups as well. Once in a while I mount one of the latest snapshots and check a bit randomly if things "look ok". I'd like to have a better system to check if backups are okay, so any pointers would be appreciated.
On a more cloudron-related note, @girish, is there a way people test restoring backups on their cloudrons? How would one go about tesing if restore will run okay when needed?
-
OK, after 4 hours...
checkPack: Load: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 507.606314ms: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 985.229971ms: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 803.546856ms: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 1.486109007s: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 2.070709754s: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 3.67875363s: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 4.459624189s: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 6.775444383s: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 15.10932531s: The specified key does not exist. Load(<data/4a8f87b965>, 0, 0) returned error, retrying after 13.811796615s: The specified key does not exist. checkPack: Load: The specified key does not exist. [2:36:37] 100.00% 57800 / 57800 packs Fatal: repository contains errors -
@necrevistonnezr how have you been testing backups? I have a raspberry pi at home, set up with restic/rclone backups as well. Once in a while I mount one of the latest snapshots and check a bit randomly if things "look ok". I'd like to have a better system to check if backups are okay, so any pointers would be appreciated.
On a more cloudron-related note, @girish, is there a way people test restoring backups on their cloudrons? How would one go about tesing if restore will run okay when needed?
@malvim said in Backup Improvements: Restic Backend:
On a more cloudron-related note, @girish, is there a way people test restoring backups on their cloudrons? How would one go about tesing if restore will run okay when needed?
Yes, use dry-run - https://docs.cloudron.io/backups/#dry-run
For Cloudron 9, we are adding backup integrity checks feature . This will prevent against bitrot and also some sort of self validation that it is backing up all files.
-
wrote on Feb 5, 2025, 6:30 PM last edited by
Oh, nice. I'll try dry-run, then, thanks.
Integrity checks sound like a great feature, looking forward to it.
Thanks!
-
@necrevistonnezr how have you been testing backups? I have a raspberry pi at home, set up with restic/rclone backups as well. Once in a while I mount one of the latest snapshots and check a bit randomly if things "look ok". I'd like to have a better system to check if backups are okay, so any pointers would be appreciated.
On a more cloudron-related note, @girish, is there a way people test restoring backups on their cloudrons? How would one go about tesing if restore will run okay when needed?
wrote on Feb 5, 2025, 6:50 PM last edited by@malvim said in Backup Improvements: Restic Backend:
@necrevistonnezr how have you been testing backups? I have a raspberry pi at home, set up with restic/rclone backups as well. Once in a while I mount one of the latest snapshots and check a bit randomly if things "look ok". I'd like to have a better system to check if backups are okay, so any pointers would be appreciated.
https://restic.readthedocs.io/en/latest/045_working_with_repos.html#checking-integrity-and-consistency
and, as I said, spot checks for files with restic-browser.