Unsolved [BUG] Automount fail on reboot with Ubuntu 20.04

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Looks like we have to debug this further then on fresh installations to get some more information what systemd does differently in both scenarios.

I'll run an fresh install of Cloudron than install davfs2 just to be sure

[JOduMonT]

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

I'll run an fresh install of Cloudron than install davfs2 just to be sure

so boot up a new instance Ubuntu 20.04 LTS @Hetzner
ran

wget https://cloudron.io/cloudron-setup
chmod +x ./cloudron-setup
./cloudron-setup

reboot than install davfs2 apt install -y davfs2
and configure /etc/fstab and /etc/davfs2/secrets
reboot

it don't mount automatically
but mount without issue with mount -a

[nebulon]

@jodumont I was able to reproduce this now also outside of hetzner on 20.04...not yet sure why and what causes the difference

[nebulon]

To give some update, this is DNS related and how the init sequence works on 20.04 now.
Problem is at the point when systemd decides to attempt to mount the remote filesystems, unbound, the dns resolver is not yet started. This means the remote fs cannot be mounted.

There are currently two workarounds:

1. Instead of using the DNS name in the fstab entry, just use the IP
2. add x-systemd.automount as an additional argument for the mountpoint in the fstab entry

Ideally we find a better flow by tweaking some of the init order in the future.

For now I've added that option requirement at https://docs.cloudron.io/backups/#cifs

[robi]

@nebulon what about a retry after the initial failure after unbound loads?

maybe add a mount -a at the end of the unbound script?

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Instead of using the DNS name in the fstab entry, just use the IP

I'm old school, I prefer IP
So it is probably related to unbound-resolvconf no ?:

Also on my side, by default my Hetzner NAS return me an IPv6, I didn't even know my Cloudron box as an IPv6

it is the same on your side ?

[JOduMonT]

@robi said in [BUG] Automount fail on reboot with Ubuntu 20.04:

@nebulon what about a retry after the initial failure after unbound loads?
maybe add a mount -a at the end of the unbound script?

How disabling the IPv6 in Cloudron

[girish]

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

So it is probably related to unbound-resolvconf no ?:

Issue is related to Cloudron. We have an internal DNS server (unbound) and it's configured in such a way that it has to start after docker (very tricky to make it start before docker). Unfortunately, because the DNS starts only after docker, it's a bit too late for services like network mounts which start before them. Which is why changing the mount from name based to IP makes it all work.

I guess the fix is to change the way DNS server starts up but this is quite a complex task.

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Instead of using the DNS name in the fstab entry, just use the IP

• with IPv6 I have this error: /sbin/mount.davfs: invalid URL
• with IPv4 I have this error: /sbin/mount.davfs: Mounting failed. 301 Moved Permanently

[girish]

@jodumont Oh.. maybe davfs requires the hostname because of vhost based configs! Can you try adding x-systemd.automount into the fstab entry instead?

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

add x-systemd.automount as an additional argument for the mountpoint in the fstab entry

adding x-systemd.automount, in /etc/fstab after _netdev, work well

Thanks for all of you (but specially @nebulon); without your help, I would probably be crying in a corner

[JOduMonT]

@girish said in [BUG] Automount fail on reboot with Ubuntu 20.04:

@jodumont Oh.. maybe davfs requires the hostname because of vhost based configs! Can you try adding x-systemd.automount into the fstab entry instead?

it worked for one or two reboot, than I started adding Volumes in my Cloudron (not really sure if it related) and I noticed it stop.

• not on Cloudron, on my machine (but also Ubuntu 20.04LTS) it work well if I add user,noauto than mount it as a user.
• on proxmox (Debian 10) it work well with x-system-d.automount

Now CIFS or DAVFS2 with the option x-systemd.automount and even after I deleted all my Volumes in Cloudron WebUI have to same behavior (I need to login and do a mount -a)

[robi]

The simpler solution would be to add the domains for mounts into the /etc/hosts file so no resolution is required.

[JOduMonT]

@robi said in [BUG] Automount fail on reboot with Ubuntu 20.04:

The simpler solution would be to add the domains for mounts into the /etc/hosts file so no resolution is required.

where were you ?
it is interesting more the technology become complex
more we forget about simple solution which was the norm 30 years ago

FYI: I add both IPv6 and IPv4

[robi]

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

where were you ?

I am here. Funny you should mention it.
Lol, I went to get some pizza today and got inspired.
Sometimes all we need is a break and some perspective.
Maybe that's a superpower.

FYI: I add both IPv6 and IPv4

Well done. Curious which it actually uses to connect.

[msbt]

the problem with the storagebox might be an ip change, hetzner specifically states

It is very important to use the DNS name (.your-storagebox.de) instead of the IP address for your Storage Box; this is because the IP address can change. With the DNS address, you can access your Storage Box via IPv4 and IPv6.

[robi]

@msbt v4 addresses changing I can understand, but v6 generally don't change much at all.

Also odd that a storage box would have dynamic addressing. Is there any additional info on why it's not static?

[nebulon]

Using remote mounts also with volumes does add some possible race and inconsistency it looks like. So on Cloudron docker has to be started before unbound, now the remote mounting requires unbound for DNS resolving, however depending on how fast docker containers come up, they require volume mounts, which in turn, if depending on the remote mount points, we end up with some circular dependency.

The systemd automount would attempt to mount a remote once anything accesses the mountpoint. So if docker is quick enough and you have assigned some volumes from a mountpoint for a container, then again it would attempt to mount this before unbound is working.

For a start the automount will solve the issue of using a remote storage as a backup storage. When used with volumes, I think only using IPs will work for now.

[JOduMonT]

@msbt said in [BUG] Automount fail on reboot with Ubuntu 20.04:

the problem with the storagebox might be an ip change, hetzner specifically states

they obviously say that simply to cover their ass in case of an power interruption, I don't really see when that would happen in other case.

@msbt I'm curious to read the whole chapter, where did you found this info ?

[JOduMonT]

@robi said in [BUG] Automount fail on reboot with Ubuntu 20.04:

I am here. Funny you should mention it.
Lol, I went to get some pizza today and got inspired.

I eat pizza last tonight, still not really inspired, but I installed Windows

[msbt]

@jodumont the word "hetzner" was a link to the docs where this is mentioned: https://docs.hetzner.com/robot/storage-box/general

[JOduMonT]

@nebulon said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Using remote mounts also with volumes does add some possible race and inconsistency it looks like

Not on the Cloudron side, but remotely NAS is in Germany, I'm in Bangkok I feel it and even for backup sometime the NAS is not available.

[JOduMonT]

@jdaviescoates said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Streaming music off of my Hetzner Storage Box mounted as a Cloudron Volume and connected to my Navidrome app as I type.

How it goes actually,
I'm trying to mount some directory from the NAS as volumes in CLOUDRON but most of the time apps (mainly tried with Nextcloud) start before so then it mess everything up
For now I mount it directly from Nextcloud interface but I can't share it with others apps.
My goal would be

1. saving money since 1TB is 10€ instead of 40€ if you build a volume of 1TB
2. but also make apps interactive/sharing space such as: sync/uploading files with Nextcloud and make it available in Navidrome or Jellyfin
3. I, aesthetically don't like to see the sharing arrow
   

@nebulon I wonder if it would be possible to transfer the mount volume principle inside the docker, at the container level ?
such as every docker will be the samba/davfs2 client and retain the info in their /etc/fstab ??

[girish]

@jodumont I think you can raise this issue in Nextcloud forum to change the icon. TBH, I am not entirely sure why they are different icons. What does a normal user have to gain by knowing the distinction between a volume and a normal folder? Also, the arrow seems to suggest that it's going to pop up a new window or something, which I assume it doesn't.

[JOduMonT]

@girish fair for the icons, and good point about open in a new tab
but what about the sharing volume in Cloudron ?
do you think would make sense to do the apps level or it will become too confusing and heavy on the system ?

[marcusquinn]

Those folder icons look like they are because they are shared folders.

[girish]

@marcusquinn Oh really, then those arrows look even more confusing. Usually that icon (with the arrow) is associated with "sharing action" (i.e clicking it lets you share something) and not meant to indicate "shared items". But I am not a UI expert.

[JOduMonT]

@girish and @nebulon, so do you think it is just me misusing the purpose of NAS / Volume in Cloudron or, I could have hope about this change ?

@jodumont said in [BUG] Automount fail on reboot with Ubuntu 20.04:

but what about the sharing volume in Cloudron ?
do you think would make sense to do the apps level or it will become too confusing and heavy on the system ?

[girish]

I will mark this as unsolved as per https://forum.cloudron.io/post/24684 . Currently, there is no solution to use DNS names in mounts, we have to rework our DNS system a bit to make it work.

[JOduMonT]

@girish said in [BUG] Automount fail on reboot with Ubuntu 20.04:

Currently, there is no solution to use DNS names in mounts

don't worry, I understand that is an exception and probably not a priority, thanks for the consideration