Unbound Service not working

[nebulon]

Does unbound itself report any errors? Check journalctl -u unbound

[skeats]

Sep 28 05:01:44 cloudron unbound[819]: [819:0] info: generate keytag query _ta-4f66. NULL IN
Sep 28 05:30:33 cloudron systemd[1]: Stopping unbound.service - Unbound DNS Resolver...
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: service stopped (unbound 1.19.2).
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: server stats for thread 0: 1353 queries, 288 answers from cache, 1065 recursions, 0 prefetch, 0 rejected by ip ra>
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: server stats for thread 0: requestlist max 15 avg 0.586854 exceeded 28 jostled 0
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: average recursion processing time 0.108782 sec
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: histogram of recursion processing times
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: [25%]=0.0432723 median[50%]=0.0667963 [75%]=0.117063
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: lower(secs) upper(secs) recursions
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.008192 0.016384 45
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.016384 0.032768 95
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.032768 0.065536 372
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.065536 0.131072 338
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.131072 0.262144 112
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.262144 0.524288 43
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 0.524288 1.000000 28
Sep 28 05:30:33 cloudron unbound[819]: [819:0] info: 1.000000 2.000000 4
Sep 28 05:30:33 cloudron systemd[1]: unbound.service: Deactivated successfully.
Sep 28 05:30:33 cloudron systemd[1]: Stopped unbound.service - Unbound DNS Resolver.
Sep 28 05:30:33 cloudron systemd[1]: unbound.service: Consumed 2.639s CPU time, 15.4M memory peak, 0B memory swap peak.
Sep 28 05:30:33 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 28 05:30:33 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 28 05:30:33 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 28 05:30:33 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 1.
Sep 28 05:30:34 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 28 05:30:34 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 2.
Sep 28 05:30:34 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 28 05:30:34 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 3.
Sep 28 05:30:34 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 28 05:30:34 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 4.
Sep 28 05:30:34 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 28 05:30:34 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 28 05:30:34 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 28 05:30:35 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
Sep 28 05:30:35 cloudron systemd[1]: unbound.service: Start request repeated too quickly.
Sep 28 05:30:35 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 28 05:30:35 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
-- Boot 41862792df8b41d4b1f0d020a113a679 --
Sep 30 14:06:58 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:06:58 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:06:58 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:06:58 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:06:58 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 1.
Sep 30 14:06:58 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:06:58 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:06:58 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:06:58 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:06:58 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 2.
Sep 30 14:06:59 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:06:59 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 3.
Sep 30 14:06:59 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:06:59 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 4.
Sep 30 14:06:59 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:06:59 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Start request repeated too quickly.
Sep 30 14:06:59 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:06:59 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
-- Boot c896dce9e2024762ab95f16909c5b548 --
Sep 30 14:27:08 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:27:08 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 1.
Sep 30 14:27:08 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:27:08 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 2.
Sep 30 14:27:08 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
Sep 30 14:27:08 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 3.
Sep 30 14:27:08 cloudron systemd[1]: Starting unbound.service - Unbound DNS Resolver...
Sep 30 14:27:08 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
lines 1355-1404

[nebulon]

That is indeed not a lot of information here. Upon search on the web apparently exit code 203 often means that the binary/executable does not exist. Is unbound still properly installed?

[skeats]

Yes, Unbound is properly installed

[nebulon]

If you manually run unbound -d is it any more chatty about the issue?

[skeats]

It is just hanging and nothing is coming up, but I can Control C to stop it

[skeats]

I ran this and got the following:

cloudron-support --troubleshoot
Vendor: QEMU Product: Standard PC (i440FX + PIIX, 1996)
Linux: 6.8.0-45-generic
Ubuntu: noble 24.04
Processor: AMD Ryzen 7 3800X 8-Core Processor
BIOS pc-i440fx-9.0 CPU @ 2.0GHz x 8
RAM: 32866264KB
Disk: /dev/sda2 701G
[OK] node version is correct
[OK] IPv6 is enabled
[OK] docker is running
[OK] MySQL is running
[OK] nginx is running
[OK] dashboard cert is valid
[OK] dashboard is reachable via loopback
[OK] box v8.0.6 is running
[OK] netplan is good
[OK] DNS is resolving via systemd-resolved
[FAIL] Could not load dashboard domain.

[skeats]

I ran this and got the following:

sudo journalctl -xeu unbound.service
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit unbound.service has begun execution.
░░
░░ The job identifier is 2602.
Sep 30 19:09:37 cloudron systemd[1]: unbound.service: Control process exited, code=exited, status=203/EXEC
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ An ExecStartPre= process belonging to unit unbound.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 203.
Sep 30 19:09:37 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit unbound.service has entered the 'failed' state with result 'exit-code'.
Sep 30 19:09:37 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
░░ Subject: A start job for unit unbound.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit unbound.service has finished with a failure.
░░
░░ The job identifier is 2602 and the job result is failed.
Sep 30 19:09:38 cloudron systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ Automatic restarting of the unit unbound.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Sep 30 19:09:38 cloudron systemd[1]: unbound.service: Start request repeated too quickly.
Sep 30 19:09:38 cloudron systemd[1]: unbound.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit unbound.service has entered the 'failed' state with result 'exit-code'.
Sep 30 19:09:38 cloudron systemd[1]: Failed to start unbound.service - Unbound DNS Resolver.
░░ Subject: A start job for unit unbound.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit unbound.service has finished with a failure.
░░
░░ The job identifier is 2791 and the job result is failed.

[girish]

@skeats I think this is because unbound-anchor package is not installed. Can you apt install unbound-anchor ? After that, systemctl restart unbound should work.

This mistake happened because we missed he installation of that package in the upgrade guide. It's been since added at https://docs.cloudron.io/guides/upgrade-ubuntu-24/#post-upgrade

[skeats]

Hi @girish,

I did that step and did a server reboot as the systemctl restart unbound did not work and when it came back up, all was good again! Thank you!