All Cloudron Services down on my instance

[Kubernetes]

Since I tried to do some automagic with ip-address lists sent to cloudron API for firewall I broke my cloudron.

cloudron-support --troubleshoot :

root@cloudron:~# cloudron-support --troubleshoot
Vendor: Hetzner Product: vServer
Linux: 6.8.0-94-generic
Ubuntu: noble 24.04
Execution environment: kvm
Processor: Intel Xeon Processor (Skylake, IBRS, no TSX)
BIOS NotSpecified  CPU @ 2.0GHz x 8
RAM: 15988568KB
Disk: /dev/sda1       118G
[OK]	node version is correct
[FAIL]	Server has an IPv6 address but api.cloudron.io is unreachable via IPv6 (ping6 -q -c 1 api.cloudron.io)
Instead of disabling IPv6 globally, you can disable it at an interface level.
	sysctl -w net.ipv6.conf.eth0.disable_ipv6=1
For the above configuration to persist across reboots, you have to add below to /etc/sysctl.conf
	net.ipv6.conf.eth0.disable_ipv6=1

cloudron-support --check-services:

root@cloudron:~# cloudron-support --check-services
[FAIL]	Service 'mysql' is not reachable
[FAIL]	Service 'postgresql' is not reachable
[FAIL]	Service 'mongodb' is not reachable
[FAIL]	Service 'mail' is not reachable
[FAIL]	Service 'graphite' is not reachable
[FAIL]	Service 'sftp' is not reachable

When I reboot the machine, the services comming up, but down again within 1-2 Minutes.

Any advice how to fix that? As this is urgent for me, I would be happy to get quick help. Thank you!

[james]

Hello @kubernetes
What is this automagic with ip-address lists sent to cloudron API for firewall?
A detailed step-by-step procedure how you ended up in this situation is needed to give you meaningful support.

[Kubernetes]

Thanks for quick reply @james

Its a shell script that is using POST https://CLOUDRON/api/v1/network/blocklist to block IPs from lists like:
https://iplists.firehol.org/files/spamhaus_drop.netset
https://iplists.firehol.org/files/spamhaus_edrop.netset
https://iplists.firehol.org/files/firehol_level1.netset
https://lists.blocklist.de/lists/mail.txt

I assume that one of those lists did contain something that did break my cloudron instance. The POST-request did timedout, and when I was looking to my Cloudron I saw that the services are down.

Luckily I was able to delete the list of IPs from Firewall via GUI after restarting of Cloudron in the first 1-2 minutes. Since then my Cloudron is running good again. I will investigate further on my end if there was something in the ip-list that may break cloudron and will report if there is anything wrong on cloudron or on my script.

[Kubernetes]

Since my clouron is running good again, I found some ip-addresses in the mentioned lists that may have forced cloudron firewall to block cloudron itself... the issue is solved.

[Kubernetes]

Maybe just one more comment. During troubleshooting I found out that with cloudron-support it is possible to use --disable-ipv6 , but re-enablement is not available.

[joseph]

@kubernetes if it helps, the internet network is documented here - https://docs.cloudron.io/network#internal-network

[joseph]

@kubernetes also --reenable-ipv6

[Kubernetes]

@joseph I know, but it failed

root@cloudron:~# cloudron-support --reenable-ipv6
/usr/bin/cloudron-support: line 932: reenable_ipv6_persistent: command not found

[girish]

@Kubernetes my bad, typo... fixed now.