1.1.1.1 and 8.8.8.8 did the trick.
... because my pihole wasn't able to do ipv6 dns resolution? which must have become a requirement from 7.7.2 to 8.0.3? As mentioned earlier I don't have any internet ipv6 connectivity so I'm not entirely sure how this is happening... but I'm not changing anything now that things are working.
For people finding the same issue the solution I did was to fresh install ubuntu server, set static IP and dns to 1.1.1.1 and 8.8.8.8, I then restored cloudron as normal.
They are my two piholes. I will reinstall with DNS=8.8.8.8 1.1.1.1 but I doubt this is the issue.
resolvectl after changing /etc/systemd/resolved.conf. Only change is Global.
Global
Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Current DNS Server: 192.168.0.5
DNS Servers: 192.168.0.5 192.168.0.6
Log:
box:tasks startTask: 7791 done. error: { stack: 'BoxError: Network error downloading icon : \n' + ' at /home/yellowtent/box/src/appstore.js:493:33\n' + ' at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' + ' at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n' + ' at async Object.downloadIcon (/home/yellowtent/box/src/appstore.js:487:12)\n' + ' at async downloadIcon (/home/yellowtent/box/src/apptask.js:208:26)\n' + ' at async configure (/home/yellowtent/box/src/apptask.js:564:5)', name: 'BoxError', reason: 'Network Error', details: {}, message: 'Network error downloading icon : ' }
systemctl status systemd-resolved: yes it's running
Results of resolvectl:
Global
Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Link 2 (enp1s0)
Current Scopes: DNS
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.0.5
DNS Servers: 192.168.0.5 192.168.0.6
Link 3 (wlp2s0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 4 (docker0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 5 (br-60558136c824)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 7 (veth14a60e8)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 9 (veth781f0fd)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 11 (veth6430967)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 13 (veth1baaf71)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 15 (vethf12b789)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 17 (veth41c82ea)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 19 (veth23598f8)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 21 (veth279bdec)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 23 (veth6e84b1b)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 25 (veth305c936)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
GNU nano 6.2 /etc/resolv.conf
nameserver 127.0.0.53
options edns0 trust-ad
search .
Worked for installing 7.7.2. Upgrading to 8.0.3 resulting in the same old downloading icon error.
I can try that next yes.
First attempt failed.
What I did in the terminal
<snip>:~$ wget https://cloudron.io/cloudron-setup
--2024-08-20 10:33:02-- https://cloudron.io/cloudron-setup
Resolving cloudron.io (cloudron.io)... 165.227.67.76, 2604:a880:800:10::b66:f001
Connecting to cloudron.io (cloudron.io)|165.227.67.76|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12646 (12K) [application/octet-stream]
Saving to: ‘cloudron-setup’
cloudron-setup 100%[===================>] 12.35K --.-KB/s in 0s
2024-08-20 10:33:03 (426 MB/s) - ‘cloudron-setup’ saved [12646/12646]
<snip>:~$ chmod +x cloudron-setup
<snip>:~$ sudo ./cloudron-setup --version 7.7.2
[sudo] password for <snip>:
##############################################
Cloudron Setup (7.7.2)
##############################################
Follow setup logs in a second terminal with:
$ tail -f /var/log/cloudron-setup.log
Join us at https://forum.cloudron.io for any questions.
=> Updating apt and installing script dependencies
=> Validating setup token
=> Checking version
=> Downloading Cloudron version 7.7.2 ...
=> Installing base dependencies (this takes some time) ...Init script failed. Se e /var/log/cloudron-setup.log for details
/var/log/cloudron-setup.log
Setting up mysql-server-8.0 (8.0.39-0ubuntu0.22.04.1) ...
update-alternatives: using /etc/mysql/mysql.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Renaming removed key_buffer and myisam-recover options (if present)
mysqld will log errors to /var/log/mysql/error.log
mysqld is running as pid 5694
Created symlink /etc/systemd/system/multi-user.target.wants/mysql.service → /lib/systemd/system/mysql.service.
Setting up unbound (1.13.1-1ubuntu5.5) ...
Created symlink /etc/systemd/system/multi-user.target.wants/unbound.service → /lib/systemd/system/unbound.service.
Created symlink /etc/systemd/system/unbound.service.wants/unbound-resolvconf.service → /lib/systemd/system/unbound-resolvconf.service.
Setting up cifs-utils (2:6.14-1ubuntu0.1) ...
update-alternatives: using /usr/lib/x86_64-linux-gnu/cifs-utils/idmapwb.so to provide /etc/cifs-utils/idmap-plugin (idmap-plugin) in auto mode
Setting up gcc (4:11.2.0-1ubuntu1) ...
Setting up libgd3:amd64 (2.3.0-2ubuntu2) ...
Setting up libstdc++-11-dev:amd64 (11.4.0-1ubuntu1~22.04) ...
Setting up g++-11 (11.4.0-1ubuntu1~22.04) ...
Setting up libnginx-mod-http-image-filter (1.18.0-6ubuntu14.4) ...
Setting up g++ (4:11.2.0-1ubuntu1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up build-essential (12.9ubuntu3) ...
Setting up nginx-core (1.18.0-6ubuntu14.4) ...
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.
invoke-rc.d: initscript nginx, action "start" failed.
^[[0;1;31m×^[[0m nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (^[]8;;file://nuccloudronv2/lib/systemd/system/nginx.service^G/lib/systemd/system/nginx.service^[]8;;^G; enabled; vendor preset: enabled)
Active: ^[[0;1;31mfailed^[[0m (Result: exit-code) since Tue 2024-08-20 10:34:28 UTC; 23ms ago
Docs: ^[]8;;man:nginx(8)^Gman:nginx(8)^[]8;;^G
Process: 6179 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; ^[[0;1;31m(code=exited, status=1/FAILURE)^[[0m
CPU: 5ms
Aug 20 10:34:28 nuccloudronv2 systemd[1]: Starting A high performance web server and a reverse proxy server...
Aug 20 10:34:28 nuccloudronv2 nginx[6179]: nginx: [emerg] socket() [::]:80 failed (97: Unknown error)
Aug 20 10:34:28 nuccloudronv2 nginx[6179]: nginx: configuration file /etc/nginx/nginx.conf test failed
Aug 20 10:34:28 nuccloudronv2 systemd[1]: ^[[0;1;39m^[[0;1;31m^[[0;1;39mnginx.service: Control process exited, code=exited, status=1/FAILURE^[[0m
Aug 20 10:34:28 nuccloudronv2 systemd[1]: ^[[0;1;38;5;185m^[[0;1;39m^[[0;1;38;5;185mnginx.service: Failed with result 'exit-code'.^[[0m
Aug 20 10:34:28 nuccloudronv2 systemd[1]: ^[[0;1;31m^[[0;1;39m^[[0;1;31mFailed to start A high performance web server and a reverse proxy server.^[[0m
dpkg: error processing package nginx-core (--configure):
installed nginx-core package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of nginx-full:
nginx-full depends on nginx-core (>= 1.18.0-6ubuntu14.4); however:
Package nginx-core is not configured yet.
nginx-full depends on nginx-core (<< 1.18.0-6ubuntu14.4.1~); however:
Package nginx-core is not configured yet.
dpkg: error processing package nginx-full (--configure):
dependency problems - leaving unconfigured
Processing triggers for libc-bin (2.35-0ubuntu3.8) ...
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for ufw (0.36.1-4ubuntu0.1) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for resolvconf (1.84ubuntu1) ...
Errors were encountered while processing:
nginx-core
nginx-full
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)
Not solved, haven't had a chance to try it yet.
ok so next thing to try...
Install Ubuntu Server 22.04 LTS
Remove ipv6 via grub BEFORE installing cloudron
Install and restore cloudron to working 7.7.2
Upgrade to 8.0.3... and hope things fall back to ipv4 like they should
Is there any other steps I could take to try and get this working?
Without adding --ipv6 it works fine once it falls back to ipv4.
Adding --ipv6:
curl -v https://api.cloudron.io/api/v1/apps/com.openwebui.cloudronapp/versions/2.3.10/icon --ipv6
Really don't think it's DNS, I have a lot of other systems (15+) that are working perfectly fine in this environment.
Trying to get to the bottom of this regression and solve it, having my cloudron stuck on an old version is not a long term solution. What's the command line equivalent of Cloudron downloading the icon?
Cloudron on ipv4 is static ip and only uses pihole for DNS. I can set the dns server to be something else but that totally isn't the problem here.
Ubuntu Server 22.04 LTS on 7.7.2, without ipv6 = fine
Upgrade to 8.0.3 in same environment = apps don't launch with initially reported error
My pihole is setting up (dhcp) ipv6, which I switched on trying to get ipv6 to work. I can turn it back off.
my HTPC and ROG Ally both have ipv6 addresses and can ping local ipv6. As I said though, my ISP does not support/offer ipv6 so any connection to the internet can only be ipv4.
@nebulon
I tried the GRUB method here: https://itsfoss.com/disable-ipv6-ubuntu-linux/#2-disable-ipv6-using-grub
Still same issue. Is there a better or recommended way to disable ipv6?
As you've probably noted, 7.7.2 worked with no problems at all, it's only after it updated to 8.0.3
Unfortunately ipv6 is not available for my internet service. This is not an isolated thing, after some quick searching there are plenty of ISPs still around that don't do IPv6.
ok so investigation has discovered... ipv6 not enabled by default on my ISP which is most likely the issue. I have changed ISP since I last had an ipv6 issue with Cloudron.
On the phone to get it resolved atm.
As a note, because the 8.0.3 backups weren't proper the restore I did was to the 7.7.2.
Another thing I could do is use a static ipv6 address? I'd need to figure out what that would be as I am unfamiliar with ipv6.
Not solved unfortunately. It updated to 8.0.3 this morning and everything is broken again in the same way.
Cloudron is run on a dedicated NUC in my house (selfhost). Running Ubuntu Server 22.04.4 LTS. Configuration is very simple, just going through the prompts and then immediately install cloudron once I get SSH, but I noticed that it didn't pull an ipv6 address when going through the initial config.
Either my DHCPv6 is broken or it's the problem I found here:
https://ubuntuforums.org/showthread.php?t=2487465
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2025382
I will try the sudo dhclient -6 <interface> fix when I get home, I'm out of time for today and need to rush off to work.
