All Apps Error Status After Update to 8.0.3

[nebulon]

Looks like from the initial attempt of curl using ipv6 and failing, that ipv6 is not working correctly on your system and it falls back to ipv4. This likely causes various issues around the system. Can you double check your ipv6 setup?

Basically if you add --ipv6 it should always fail until ipv6 connections are routed correctly.

[playsia]

ipv6 is no longer working. Note that it was working before because I remember having to enable it because of an update issue in the past.

I'm going to attempt a backup and re-install.

Just went to backup and my backups are now 23mb instead of ~46GB. The apps are still taking up space in system info.

[nebulon]

It should be possible to fixup ipv6 in the current system. A wholesome restore will most likely not be required.

[playsia]

Should be possible yes. I tried a few things but got nowhere and I have limited time to fiddle with things atm. Restoration has everything working again.

Hoping the update doesn't mess up again...

[nebulon]

Glad the restore did work for you. Which VPS provider are you using, as network setup is not really part of Cloudron, but usually done by the server provider cloud-init scripts.

[playsia]

Not solved unfortunately. It updated to 8.0.3 this morning and everything is broken again in the same way.

Cloudron is run on a dedicated NUC in my house (selfhost). Running Ubuntu Server 22.04.4 LTS. Configuration is very simple, just going through the prompts and then immediately install cloudron once I get SSH, but I noticed that it didn't pull an ipv6 address when going through the initial config.

Either my DHCPv6 is broken or it's the problem I found here:
https://ubuntuforums.org/showthread.php?t=2487465
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2025382

I will try the sudo dhclient -6 <interface> fix when I get home, I'm out of time for today and need to rush off to work.

[playsia]

As a note, because the 8.0.3 backups weren't proper the restore I did was to the 7.7.2.

Another thing I could do is use a static ipv6 address? I'd need to figure out what that would be as I am unfamiliar with ipv6.

[playsia]

ok so investigation has discovered... ipv6 not enabled by default on my ISP which is most likely the issue. I have changed ISP since I last had an ipv6 issue with Cloudron.

On the phone to get it resolved atm.

[playsia]

Unfortunately ipv6 is not available for my internet service. This is not an isolated thing, after some quick searching there are plenty of ISPs still around that don't do IPv6.

[nebulon]

In such a case you probably have to disable ipv6 for the Cloudron and then things should work?

[playsia]

@nebulon
I tried the GRUB method here: https://itsfoss.com/disable-ipv6-ubuntu-linux/#2-disable-ipv6-using-grub

Still same issue. Is there a better or recommended way to disable ipv6?

As you've probably noted, 7.7.2 worked with no problems at all, it's only after it updated to 8.0.3

[nebulon]

For ipv6 as such, there is inbound and outbound connectivity. Inbound may not be routed correctly to your server directly depending on your ISP and thus the AAAA records should not be setup, to avoid clients trying this. For outbound, if your other systems like your laptop works fine, your Cloudron should also. Would be good to understand what the difference is.

[playsia]

my HTPC and ROG Ally both have ipv6 addresses and can ping local ipv6. As I said though, my ISP does not support/offer ipv6 so any connection to the internet can only be ipv4.

[nebulon]

I am not familar with what your ISP uses, but if it has no actual ipv6 connectivity, why is your router then setting up ipv6 for your local network? The German ISPs which only offer ipv6 over ipv4 make outbound ipv6 connections work, which is what seems to not work in your case. But as said Cloudron does not do any special network configuration as such, so whatever ubuntu sets up is used, so not sure how to proceed here.

[playsia]

My pihole is setting up (dhcp) ipv6, which I switched on trying to get ipv6 to work. I can turn it back off.

[necrevistonnezr]

Having a pi-hole between Cloudron and the internet is, generally speaking, not a good idea (nor required) from my personal experience. You want to block end devices, particular those you cannot really control (like TVs, mobiles, etc.) but not a server that you can control.
In particular, since you can run Adguard Home on Cloudron and have it serve the rest of the network as an effective adblocker.

[playsia]

Cloudron on ipv4 is static ip and only uses pihole for DNS. I can set the dns server to be something else but that totally isn't the problem here.

Ubuntu Server 22.04 LTS on 7.7.2, without ipv6 = fine
Upgrade to 8.0.3 in same environment = apps don't launch with initially reported error

[nebulon]

Right it is unclear here what the Cloudron update has to do really. All I can think of is the change from not using unbound anymore for local DNS resolving but now relying on whatever the environment the server runs in provides. So all in all all whatever ubuntu would setup without Cloudron should be the same as what your Cloudron should be configured with.

Maybe the previously used unbound just covered up some other networking setup issues, which are now revealed with stock ubuntu network setup.

[playsia]

Really don't think it's DNS, I have a lot of other systems (15+) that are working perfectly fine in this environment.

Trying to get to the bottom of this regression and solve it, having my cloudron stuck on an old version is not a long term solution. What's the command line equivalent of Cloudron downloading the icon?

[nebulon]

Yes something is off with the networking on that server then it seems. To replicate what the Cloudron code does here, the curl command I posted earlier is the one:

curl -v https://api.cloudron.io/api/v1/apps/com.openwebui.cloudronapp/versions/2.3.10/icon

with the mentioned flavor of adding --ipv6 to ensure curl does not automatically fallback to ipv4 if ipv6 is not working as expected. So if curl works with ipv6 and ipv4 I can't see how Cloudron wouldn't be able to make requests.