I didn't uncover the cause, but I believe it was related to the VM running out of disk space. I wound up reinstalling.

I let a large upload job run unattended and wound up completely filling the drive on the VM running Cloudron. I then cleared about 100 GB, rebooted for good measure, resized the partition using parted, expanded the filesystem, rebooted again, and let it run normally for a few days. That's when I noticed I couldn't install any new apps. I suspect that either the partition / filesystem resize or the disk being full broke something internally.

In hindsight I wish I had taken the time to dig into the logs a little more and see what was happening. But this is likely a direct result of my unusual environment fault.

@gog122 you can login at https://console.cloudron.io/#/login and cancel it there.

This was sorted out in support.

@nebulon I hit many road blocks indeed, but managed to move quite well forward
now i'm at the part of the npm rebuild and I can't do the db-migrate because I don't see any npm install being done that provides db-migrate within nodes_modules/.bin folder

I have documented the changes I had to do in the different install files to get it working, i can share this to the support email if you're interested

@nebulon I don't really know the S3 system they used

Thanks for the hint - trying that I got a more meaningful error notice. It was one of the ipdeny urls (IPv6 list for North Korea) that resulted in an empty HTML file, throwing off the ipfile format.

@im-fabian does using the psql command directly work? That will help us debug if this is mistake in postgres credentials or some pgadmin issue.

@Vortexapps this seems to be VM/deployment related. You have to tell us more about your setup / network for us to help further. nginx is running as you see in the troubleshooting output, so you have to check why it's not reachable by ip on port 443.

@Dave-Swift 7.7.2 is in the CI. So, in a day or two.

@ekevu123 good idea!

@girish Yup I posted it here, really for search results in case other people come up with the same issue !
it can be marked as solved !

I remember now. We have this issue with the DNS server (unbound) we use which doesn't work properly with just IPv6 with our configuration. We are working on getting away from unbound altogether because of various other issues, so will get to do this after that.

I apologize, I've been really busy. Ended up just being a combination of me setting conflicting DNS records and issues with Let's Encrypt ACME verification. I fixed both and everything is working properly now.

Great! Thank you so much for the detailed and helpful answer 🙏

I can confirm.Thank you!

@andreasdueren yes, usually that means the passphrase is wrong.

@Dont-Worry the apps run as containers and each container has it's own file system (completely different from the host filesysted). It is a sort of sandboxing for security.

What this setup means is that, even if your disc is attached to the machine, the app containers won't see it! It will be available on the host ubuntu itself but not to the apps. Cloudron Volumes is a mechanism to expose these discs on host to the apps. Hope that clarifies!

But if I could use it as a classic disc, I should mount it as a Cloudron volume If I understood correctly.

Right, and there is no way around not creating a volume.

You can also check apt list --upgradable | grep security if those are actually security updates.

@girish said in Firewall IP blocking: IPv6 not possible:

I guess this post was before we had IPv6 support. IPv6 is supported in the firewall by now.

Indeed 😊

For anyone coming later, Ubuntu server use's netplan for configuration via files in /etc/netplan and you should not configure networking directly via systemd-networkd .network files.

More details here.

In my case if I ran sudo cat /etc/netplan/00-installer-config.yaml I got the following:

# This is the network config written by 'subiquity' network: ethernets: enp2s0: dhcp4: true optional: true version: 2

The specific issue was that I had optional: true which I had done when I first setup this system as I originally had it setup to use wireless while doing the initial configuration and then switched to hard wired once I deployed it.

Changing optional: true to optional: false, then removing the .network file I had put in /etc/systemd/network, and then rebooting I can now see that cat /run/systemd/network/10-netplan-enp2s0.network no longer has RequiredForOnline=no.

Apparently the default is to require the network adapter to be in a routable state so now with the correct netplan configuration I get the right generated systemd-networkd configuration which properly waits for the interface to be routable before trying to mount volumes and all volumes mount as expected without intervention.

The root cause behind this issue was me making a configuration change I meant to be temporary but then forgot to reverse and then because I didn't understand netplan my troubleshooting was in the wrong places.