Server occasionally hard-reboots, only DNS timeouts in the log (Cloudron 9.2.0)

[archos]

Hi,

I'm running Cloudron 9.2.0 on a Hetzner dedicated server (Ubuntu 24.04, 64 GB RAM, around 107 containers). The server occasionally hard-reboots on its own and I can't figure out why.

The only thing I see in the log before the crash is a lot of these errors, which pile up for a few minutes and then the server dies:

dockerd: [resolver] failed to query external DNS server ... i/o timeout
question=";191.101.249.180.in-addr.arpa. IN PTR"

It's the same thing over and over, mostly reverse DNS lookups of IP addresses. No kernel panic, no out-of-memory, disks and RAM are fine. The system is healthy right now — the DNS flood only happens in the minutes leading up to a crash.

Any idea what could be causing this and how to stop it?

Troubleshoot output:

Cloudron: 9.2.0
Ubuntu: noble 24.04
Linux: 6.8.0-136-generic
Processor: 13th Gen Intel(R) Core(TM) i9-13900

[OK] Root disk usage is OK (43%)
[OK] Memory usage is OK (41%)
[OK] Clock is NTP-synchronized
[OK] docker is running
[OK] MySQL is running
[OK] DNS is resolving via systemd-resolved
[OK] unbound is running
[OK] nginx is running
[OK] box v9.2.0 is running
======== Summary ========
PASS: 30   WARN: 0   FAIL: 0   SKIP: 1

Thanks!

[nebulon]

If by hard reboot you mean an actual powercycle which is not driven by a kernel issues reboot, then this likely is more of a hardware/infra problem. For a start check the system messages from before the last boot (if that was one of the hard resets):

sudo journalctl --system --boot=-1

Maybe there is some kernel panic causing it, otherwise if this is software/Cloudron related then you would see "Reached target Shutdown" in the journal.

[archos]

@nebulon Thanks for the reply!
I wanted to check here first before opening a hardware ticket with Hetzner, to make sure I wasn't looking in the wrong place and it wasn't a Cloudron issue.
I've already checked journalctl --system --boot=-1 thoroughly. There is no "Reached target Shutdown" — the journal cuts off abruptly mid-operation (normal containerd shim cleanup messages, then nothing — as if someone pulled the power plug). No kernel panic, no oops, /sys/fs/pstore is empty after each freeze.
The Jul 16 crash was confirmed by Hetzner support as a hard freeze: "no video signal, no keystrokes" on KVM. They had to do a physical hardware reset.
So it does look like a hardware issue rather than Cloudron. I'll follow up with Hetzner. Thanks for helping me confirm that!

[archos]

Quick update: Hetzner offered to replace the server
but keep my drives (new motherboard, CPU, RAM). The
replacement might have a different CPU (e.g. AMD
instead of Intel) or a different board.

Could this cause any issues with Cloudron? I'm
thinking the network interface name might change
(enp5s0 → something else). Is there anything else
I should watch out for?

[nebulon]

this should all just work, if you have configured Cloudron for some reason to explicitly use some network interface you may have to reconfigure that, but this is only really used for public IP detection, so worst case you have to first adjust the DNS A record for the dashboard domain manually once to reconfigure that and resync dns from the cloudron dashboard, but very unlikely that this happens.

Further of course if you have attached disks and used that as volumes in Cloudron, if the disk identifiers change (also unlikely though) this might cause some issue, but we can adress that then.