[BUG] Automount fail on reboot with Ubuntu 20.04

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Cloudron itself does not handle any mount points as such. So this looks like this is some issue with 20.04 server then. Note that the desktop flavor has a lot more things installed usually, which may or may not trigger automounting correctly.

I have no doubt of the quality of coding from the Cloudron Team
but I just boot an ubuntu 20.04 LTS at Hetzner
did

apt update
apt install -y davfs2
mkdir /mnt/storagebox

than cut and past my 2 lines
1 from /etc/fstab
1 for /etc/davfs2/secret

reboot and it work

so yes Ubuntu Desktop have probably more fuse than server but now the only difference is Cloudron.

[nebulon]

thanks for testing, so then probably one of the dependencies we install, somehow changes either the init order or even disables some bits there. Looks like we have to debug this further then on fresh installations to get some more information what systemd does differently in both scenarios.

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Looks like we have to debug this further then on fresh installations to get some more information what systemd does differently in both scenarios.

I'll run an fresh install of Cloudron than install davfs2 just to be sure

[JOduMonT]

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

I'll run an fresh install of Cloudron than install davfs2 just to be sure

so boot up a new instance Ubuntu 20.04 LTS @Hetzner
ran

wget https://cloudron.io/cloudron-setup
chmod +x ./cloudron-setup
./cloudron-setup

reboot than install davfs2 apt install -y davfs2
and configure /etc/fstab and /etc/davfs2/secrets
reboot

it don't mount automatically
but mount without issue with mount -a

[nebulon]

@jodumont I was able to reproduce this now also outside of hetzner on 20.04...not yet sure why and what causes the difference

[nebulon]

To give some update, this is DNS related and how the init sequence works on 20.04 now.
Problem is at the point when systemd decides to attempt to mount the remote filesystems, unbound, the dns resolver is not yet started. This means the remote fs cannot be mounted.

There are currently two workarounds:

1. Instead of using the DNS name in the fstab entry, just use the IP
2. add x-systemd.automount as an additional argument for the mountpoint in the fstab entry

Ideally we find a better flow by tweaking some of the init order in the future.

For now I've added that option requirement at https://docs.cloudron.io/backups/#cifs

[robi]

@nebulon what about a retry after the initial failure after unbound loads?

maybe add a mount -a at the end of the unbound script?

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Instead of using the DNS name in the fstab entry, just use the IP

I'm old school, I prefer IP
So it is probably related to unbound-resolvconf no ?:

Also on my side, by default my Hetzner NAS return me an IPv6, I didn't even know my Cloudron box as an IPv6

it is the same on your side ?

[JOduMonT]

@robi said in [BUG] Automount fail on reboot with Ubuntu 20.04:

@nebulon what about a retry after the initial failure after unbound loads?
maybe add a mount -a at the end of the unbound script?

How disabling the IPv6 in Cloudron

[girish]

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

So it is probably related to unbound-resolvconf no ?:

Issue is related to Cloudron. We have an internal DNS server (unbound) and it's configured in such a way that it has to start after docker (very tricky to make it start before docker). Unfortunately, because the DNS starts only after docker, it's a bit too late for services like network mounts which start before them. Which is why changing the mount from name based to IP makes it all work.

I guess the fix is to change the way DNS server starts up but this is quite a complex task.

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Instead of using the DNS name in the fstab entry, just use the IP

• with IPv6 I have this error: /sbin/mount.davfs: invalid URL
• with IPv4 I have this error: /sbin/mount.davfs: Mounting failed. 301 Moved Permanently

[girish]

@jodumont Oh.. maybe davfs requires the hostname because of vhost based configs! Can you try adding x-systemd.automount into the fstab entry instead?

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

add x-systemd.automount as an additional argument for the mountpoint in the fstab entry

adding x-systemd.automount, in /etc/fstab after _netdev, work well

Thanks for all of you (but specially @nebulon); without your help, I would probably be crying in a corner

[JOduMonT]

@girish said in [BUG] Automount fail on reboot with Ubuntu 20.04:

@jodumont Oh.. maybe davfs requires the hostname because of vhost based configs! Can you try adding x-systemd.automount into the fstab entry instead?

it worked for one or two reboot, than I started adding Volumes in my Cloudron (not really sure if it related) and I noticed it stop.

• not on Cloudron, on my machine (but also Ubuntu 20.04LTS) it work well if I add user,noauto than mount it as a user.
• on proxmox (Debian 10) it work well with x-system-d.automount

Now CIFS or DAVFS2 with the option x-systemd.automount and even after I deleted all my Volumes in Cloudron WebUI have to same behavior (I need to login and do a mount -a)

[robi]

The simpler solution would be to add the domains for mounts into the /etc/hosts file so no resolution is required.

[JOduMonT]

@robi said in [BUG] Automount fail on reboot with Ubuntu 20.04:

The simpler solution would be to add the domains for mounts into the /etc/hosts file so no resolution is required.

where were you ?
it is interesting more the technology become complex
more we forget about simple solution which was the norm 30 years ago

FYI: I add both IPv6 and IPv4

[robi]

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

where were you ?

I am here. Funny you should mention it.
Lol, I went to get some pizza today and got inspired.
Sometimes all we need is a break and some perspective.
Maybe that's a superpower.

FYI: I add both IPv6 and IPv4

Well done. Curious which it actually uses to connect.

[msbt]

the problem with the storagebox might be an ip change, hetzner specifically states

It is very important to use the DNS name (.your-storagebox.de) instead of the IP address for your Storage Box; this is because the IP address can change. With the DNS address, you can access your Storage Box via IPv4 and IPv6.

[robi]

@msbt v4 addresses changing I can understand, but v6 generally don't change much at all.

Also odd that a storage box would have dynamic addressing. Is there any additional info on why it's not static?

[nebulon]

Using remote mounts also with volumes does add some possible race and inconsistency it looks like. So on Cloudron docker has to be started before unbound, now the remote mounting requires unbound for DNS resolving, however depending on how fast docker containers come up, they require volume mounts, which in turn, if depending on the remote mount points, we end up with some circular dependency.

The systemd automount would attempt to mount a remote once anything accesses the mountpoint. So if docker is quick enough and you have assigned some volumes from a mountpoint for a container, then again it would attempt to mount this before unbound is working.

For a start the automount will solve the issue of using a remote storage as a backup storage. When used with volumes, I think only using IPs will work for now.