Backup failing (flaky CIFS mount)
-
@jdaviescoates Yes, at some point we want to make it possible to add mounts easily. When we do that, we can attempt to re-mount automatically, I guess.
wrote on Dec 30, 2020, 1:00 AM last edited by@girish not sure this is the right context..
this is about existing mounts and their resilience to disconnects.
- monitoring mounts for health
- allowing for self-healing if it exists from the protocol
- if not, try harder, remount command for example
- if still not, try unmount, then remount
- if still not, notify to reboot
-
@girish not sure this is the right context..
this is about existing mounts and their resilience to disconnects.
- monitoring mounts for health
- allowing for self-healing if it exists from the protocol
- if not, try harder, remount command for example
- if still not, try unmount, then remount
- if still not, notify to reboot
-
wrote on Jan 5, 2021, 9:27 AM last edited by
@p44 said in Backup failing:
@robi right suggestions.
@girish I had this problem again: all backup folder this morning where unmounted...
I've had the same thing again too
-
@p44 said in Backup failing:
@robi right suggestions.
@girish I had this problem again: all backup folder this morning where unmounted...
I've had the same thing again too
@iqweb Are you also using CIFS/NFS?
-
@p44 said in Backup failing:
@robi right suggestions.
@girish I had this problem again: all backup folder this morning where unmounted...
I've had the same thing again too
@iqweb Still on Hetzner storage?
-
wrote on Jan 6, 2021, 9:09 AM last edited by
@girish said in Backup failing:
@iqweb Are you also using CIFS/NFS?
Yes - storage provider: CIFS Mount
@p44 said in Backup failing:
@iqweb Still on Hetzner storage?
Yes - Hetzner storage box
& again - I had to reboot to be able to remount.
-
@girish said in Backup failing:
@iqweb Are you also using CIFS/NFS?
Yes - storage provider: CIFS Mount
@p44 said in Backup failing:
@iqweb Still on Hetzner storage?
Yes - Hetzner storage box
& again - I had to reboot to be able to remount.
@iqweb Maybe Hetzner sometimes reboot Box instance? From my side is about one week CIFS mount is stable...
-
I also have to remount my cifs share about once a month. In my case I thought it was related to my residential home internet connection. Either way I think we have to handle such mount points better and maybe even attempt a remount automatically.
-
I also have to remount my cifs share about once a month. In my case I thought it was related to my residential home internet connection. Either way I think we have to handle such mount points better and maybe even attempt a remount automatically.
@nebulon Good idea to add a health-check to keep alive mounting
-
I also have to remount my cifs share about once a month. In my case I thought it was related to my residential home internet connection. Either way I think we have to handle such mount points better and maybe even attempt a remount automatically.
@nebulon Does cifs mounting have some retry flags? (i.e the fstab options)
-
@girish not that I have found and https://linux.die.net/man/8/mount.cifs does not mention anything.
Apparently systemd has some automounting feature https://www.freedesktop.org/software/systemd/man/systemd.automount.html which mounts the drive once the mountpoint is accessed by a process! Can't tell if it would reconnect such a mount though.
-
@girish not that I have found and https://linux.die.net/man/8/mount.cifs does not mention anything.
Apparently systemd has some automounting feature https://www.freedesktop.org/software/systemd/man/systemd.automount.html which mounts the drive once the mountpoint is accessed by a process! Can't tell if it would reconnect such a mount though.
@nebulon Maybe https://askubuntu.com/questions/1210867/remount-cifs-on-network-reconnect ? (just expands on your link)
-
@nebulon Maybe https://askubuntu.com/questions/1210867/remount-cifs-on-network-reconnect ? (just expands on your link)
@girish & @nebulon any news on this issue?
I started yesterday to backup via cifs to a Hetzner storagebox (2 backups a day), today the first backup failed due to
Command failed with exit code 1 (EPERM): df -kPT /backups_cifsI solved it with a manual:
sudo umount /backups_cifs sudo mount -aBut do I need to script this with a daily cron or is Cloudron able to recover from this somehow?
-
@girish & @nebulon any news on this issue?
I started yesterday to backup via cifs to a Hetzner storagebox (2 backups a day), today the first backup failed due to
Command failed with exit code 1 (EPERM): df -kPT /backups_cifsI solved it with a manual:
sudo umount /backups_cifs sudo mount -aBut do I need to script this with a daily cron or is Cloudron able to recover from this somehow?
@imc67 Dear, this happened because Hetzner did some maintenance on their Box Service.
Also a Cloudron restart solved problem.
-
@imc67 Dear, this happened because Hetzner did some maintenance on their Box Service.
Also a Cloudron restart solved problem.
-
@p44 actually I hit the same thing today morning, had to remount the disk. I happily ignored the heads up warning email about the maintenance from hetzner
@nebulon Yes, but not all my Cloudron installs has been affected, even if attached at the same Hetzner Box... Just restarted and all was operational again
-
@nebulon Yes, but not all my Cloudron installs has been affected, even if attached at the same Hetzner Box... Just restarted and all was operational again
-
@p44 in such cases a remount alone would be enough, no need to reboot the whole server, if that is what you did. This would then not affect downtime. We will soon implement auto-remounting in such cases.
@nebulon Yes before rebooting I tried to umount-remount but it seems it has been impossible due to an error... So I decided to restart.
About auto-remounting it will be an amazing feature
