Unbound service not starting after having Cloudron installed for years

[xdrakon]

I have used Cloudron to host my Apache guacamole for a couple of years now and I had issues a few days ago where it stopped working.

I got Cloudron to start working again but now 2 services in Cloudron won't run Mail and Unbound. I have dug through a bunch of posts already and nothing seems to fix it. I did notice that when trying to use curl or download new packages with apt also fails.

Here is the log for unbound:

May 18 15:54:59 ubuntuguac systemd[1]: unbound.service: Scheduled restart job, restart counter is at 4.
May 18 15:54:59 ubuntuguac systemd[1]: Stopped Unbound DNS Resolver.
May 18 15:54:59 ubuntuguac systemd[1]: Starting Unbound DNS Resolver...
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] notice: init module 0: subnet
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] notice: init module 1: validator
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] error: failed to read /var/lib/unbound/root.key
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] error: error reading auto-trust-anchor-file: /var/lib/unbound/root.key
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] error: validator: error in trustanchors config
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] error: validator: could not apply configuration settings.
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] error: module init for module validator failed
May 18 15:54:59 ubuntuguac unbound[42411]: [42411:0] fatal error: failed to setup modules
May 18 15:54:59 ubuntuguac systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
May 18 15:54:59 ubuntuguac systemd[1]: unbound.service: Failed with result 'exit-code'.
May 18 15:54:59 ubuntuguac systemd[1]: Failed to start Unbound DNS Resolver.
May 18 15:54:59 ubuntuguac systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
May 18 15:54:59 ubuntuguac systemd[1]: Stopped Unbound DNS Resolver.
May 18 15:54:59 ubuntuguac systemd[1]: unbound.service: Start request repeated too quickly.
May 18 15:54:59 ubuntuguac systemd[1]: unbound.service: Failed with result 'exit-code'.
May 18 15:54:59 ubuntuguac systemd[1]: Failed to start Unbound DNS Resolver.

Here is the log for Mail:

May 17 22:32:34 mv: cannot stat '/app/data/haraka-config/*.*access*': No such file or directory
May 17 22:32:34 setup_dovecot: Changing ownership
May 17 22:32:34 setup_haraka: Copying config template
May 17 22:32:34 setup_solr: Solr FTS is disabled
May 17 22:32:34 setup_spamd: Changing ownership
May 17 22:32:37 2024-05-18 03:32:37,880 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message.
May 17 22:32:37 2024-05-18 03:32:37,881 INFO Included extra file "/etc/supervisor/conf.d/dovecot.conf" during parsing
May 17 22:32:37 2024-05-18 03:32:37,881 INFO Included extra file "/etc/supervisor/conf.d/haraka.conf" during parsing
May 17 22:32:37 2024-05-18 03:32:37,881 INFO Included extra file "/etc/supervisor/conf.d/mail-service.conf" during parsing
May 17 22:32:37 2024-05-18 03:32:37,881 INFO Included extra file "/etc/supervisor/conf.d/redis.conf" during parsing
May 17 22:32:37 2024-05-18 03:32:37,881 INFO Included extra file "/etc/supervisor/conf.d/solr.conf" during parsing
May 17 22:32:37 2024-05-18 03:32:37,881 INFO Included extra file "/etc/supervisor/conf.d/spamd.conf" during parsing
May 17 22:32:37 2024-05-18 03:32:37,899 CRIT Server 'inet_http_server' running without any HTTP authentication checking
May 17 22:32:37 2024-05-18 03:32:37,899 CRIT Server 'unix_http_server' running without any HTTP authentication checking
May 17 22:32:37 2024-05-18 03:32:37,899 INFO RPC interface 'supervisor' initialized
May 17 22:32:37 2024-05-18 03:32:37,899 INFO RPC interface 'supervisor' initialized
May 17 22:32:37 2024-05-18 03:32:37,900 INFO supervisord started with pid 1
May 17 22:32:37 Starting supervisor
May 17 22:32:38 2024-05-18 03:32:38,903 INFO spawned: 'dovecot' with pid 61
May 17 22:32:38 2024-05-18 03:32:38,914 INFO spawned: 'haraka' with pid 62
May 17 22:32:38 2024-05-18 03:32:38,917 INFO spawned: 'mail-service' with pid 63
May 17 22:32:38 2024-05-18 03:32:38,954 INFO spawned: 'redis' with pid 64
May 17 22:32:38 2024-05-18 03:32:38,959 INFO spawned: 'spamd' with pid 65
May 17 22:32:39 64:C 18 May 2024 03:32:39.373 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=64, just started
May 17 22:32:39 64:C 18 May 2024 03:32:39.373 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
May 17 22:32:39 64:C 18 May 2024 03:32:39.374 # Configuration loaded
May 17 22:32:39 64:M 18 May 2024 03:32:39.464 * Running mode=standalone, port=6379.
May 17 22:32:39 64:M 18 May 2024 03:32:39.465 # Server initialized
May 17 22:32:39 64:M 18 May 2024 03:32:39.465 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
May 17 22:32:39 64:M 18 May 2024 03:32:39.465 * Ready to accept connections
May 17 22:32:39 Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203)
May 17 22:32:39 Warning: service auth { client_limit=1000 } is lower than required under max. load (1300)
May 17 22:32:39 doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203)
May 17 22:32:39 doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (1300)
May 17 22:32:40 2024-05-18 03:32:40,619 INFO success: dovecot entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
May 17 22:32:40 2024-05-18 03:32:40,620 INFO success: haraka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
May 17 22:32:40 2024-05-18 03:32:40,620 INFO success: mail-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
May 17 22:32:40 2024-05-18 03:32:40,620 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
May 17 22:32:40 2024-05-18 03:32:40,620 INFO success: spamd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
May 17 22:32:42 1=1487
May 17 22:32:42 2=8571
May 17 22:32:42 3=2448
May 17 22:32:42 loaded 9459 Public Suffixes
May 17 22:32:42 loaded TLD files:
May 17 22:32:47 Mail service endpoint listening on http://:::3000
May 17 22:32:48 Starting up Haraka version 2.8.28
May 17 22:32:48 [INFO] [-] [cloudron] Initializing queue server on port 6000
May 17 22:32:48 [INFO] [-] [core] Creating TLS server on [::0]:2465
May 17 22:32:48 [INFO] [-] [core] Loading plugin: cloudron
May 17 22:32:48 [INFO] [-] [core] Loading plugin: delay_deny
May 17 22:32:48 [INFO] [-] [core] Loading plugin: dkim_sign
May 17 22:32:48 [INFO] [-] [core] Loading plugin: dnsbl
May 17 22:32:48 [INFO] [-] [core] Loading plugin: headers
May 17 22:32:48 [INFO] [-] [core] Loading plugin: helo.checks
May 17 22:32:48 [INFO] [-] [core] Loading plugin: limit
May 17 22:32:48 [INFO] [-] [core] Loading plugin: queue/smtp_forward
May 17 22:32:48 [INFO] [-] [core] Loading plugin: rcpt_to.in_host_list
May 17 22:32:48 [INFO] [-] [core] Loading plugin: spamassassin
May 17 22:32:48 [INFO] [-] [core] Loading plugin: spf
May 17 22:32:48 [INFO] [-] [core] Loading plugin: tls
May 17 22:32:48 [INFO] [-] [core] Loading plugins
May 17 22:32:48 [INFO] [-] [core] [outbound] Loading outbound queue from /app/data/haraka-queue
May 17 22:32:48 [INFO] [-] [core] [outbound] Loading the queue...
May 17 22:32:48 [INFO] [-] [core] [outbound] [pid: undefined] 0 files in my delivery queue
May 17 22:32:48 [INFO] [-] [core] [outbound] [pid: undefined] 0 files in my load queue
May 17 22:32:48 [INFO] [-] [core] [outbound] [pid: undefined] 0 files in my temp fail queue
May 17 22:32:48 [INFO] [-] [core] getting SocketOpts for SMTPS server
May 17 22:32:48 [INFO] [-] [core] loading tls.ini
May 17 22:32:48 [INFO] [-] [limit] connected to redis://127.0.0.1:6379/4
May 17 22:32:48 [NOTICE] [-] [core] Listening on [::0]:2465
May 17 22:32:48 [NOTICE] [-] [core] Listening on [::0]:2525
May 17 22:32:48 [NOTICE] [-] [core] Listening on [::0]:2587
May 17 22:32:48 log format: DEFAULT
May 17 22:32:48 loglevel: INFO
May 17 22:32:58 2024-05-18 03:32:58,828 WARN received SIGTERM indicating exit request
May 17 22:32:58 2024-05-18 03:32:58,838 INFO waiting for dovecot, haraka, mail-service, redis, spamd to die
May 17 22:32:58 2024-05-18 03:32:58,936 INFO stopped: spamd (exit status 0)
May 17 22:32:59 2024-05-18 03:32:59,975 INFO stopped: redis (exit status 0)
May 17 22:32:59 2024-05-18 03:32:59,980 INFO stopped: mail-service (terminated by SIGTERM)
May 17 22:32:59 2024-05-18 03:32:59,999 INFO stopped: haraka (terminated by SIGTERM)
May 17 22:32:59 64:M 18 May 2024 03:32:59.970 # Redis is now ready to exit, bye bye...
May 17 22:32:59 64:M 18 May 2024 03:32:59.970 # User requested shutdown...
May 17 22:32:59 64:signal-handler (1716003179) Received SIGTERM scheduling shutdown...
May 17 22:33:01 2024-05-18 03:33:01,097 INFO reaped unknown pid 81
May 17 22:33:01 2024-05-18 03:33:01,097 INFO stopped: dovecot (exit status 0)

[girish]

@xdrakon can you try this - https://docs.cloudron.io/troubleshooting/#unbound ?

[xdrakon]

@girish I feel like I already tried that a few times to no avail but this time around it worked perfectly and got unbound and mail to start working again. Now guacamole is still stuck restarting saying "Restarting - Waiting for Cloudron to initialize"

It keeps throwing this in the box log.

May 20 00:04:20 box:apphealthmonitor app health: 0 running / 0 stopped / 1 unresponsive

[nebulon]

@xdrakon can you try to run systemctl restart box to reschedule healthchecks if they got stuck for some reason?

[xdrakon]

@nebulon Still stuck on the issue after restarting the box. I'm hoping that I'm not just being impatient with the restart so I'll continue to look for any changes in the log.

Logs:

May 20 10:07:31 box:box Received SIGTERM. Shutting down.
May 20 10:07:31 box:shell stopTask spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
May 20 10:07:31 box:tasks stopAllTasks: stopping all tasks
May 20 10:07:31 box:shell stopTask (stdout): All tasks stopped
May 20 10:07:36 box:server ==========================================
May 20 10:07:36 box:server Cloudron 7.4.0
May 20 10:07:36 box:server ==========================================
May 20 10:07:36 box:settings initCache: pre-load settings
May 20 10:07:36 box:tasks stopAllTasks: stopping all tasks
May 20 10:07:36 box:shell stopTask spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
May 20 10:07:36 box:shell stopTask (stdout): All tasks stopped
May 20 10:07:36 box:reverseproxy writeDashboardConfig: writing admin config for cloudron.colbywhitaker.me
May 20 10:07:37 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 20 10:07:37 box:cloudron onActivated: running post activation tasks
May 20 10:07:37 box:platform initializing platform
May 20 10:07:37 box:platform Updating infrastructure from 49.4.0 to 49.4.0
May 20 10:07:37 box:locker Acquired : platform_start
May 20 10:07:37 box:platform markApps: changedAddons: ["mysql","postgresql","mongodb","redis"]
May 20 10:07:37 box:services startServices: existing infra. incremental service create ["startMail","startTurn","startMysql","startPostgresql","startMongodb","startRedis","startGraphite","start"]
May 20 10:07:37 box:mail startMail: starting
May 20 10:07:37 box:mail restartMail: restarting mail container with mailFqdn:my.cloudron.colbywhitaker.me mailDomain:cloudron.colbywhitaker.me
May 20 10:07:37 box:shell stopMail exec: docker stop mail || true
May 20 10:07:44 box:shell removeMail exec: docker rm -f mail || true
May 20 10:07:44 box:shell stopMail (stderr): null
May 20 10:07:44 box:shell stopMail (stdout): mail
May 20 10:07:44 box:shell removeMail (stderr): null
May 20 10:07:44 box:shell removeMail (stdout): mail
May 20 10:07:44 box:mail createMailConfig: generating mail config with my.cloudron.colbywhitaker.me
May 20 10:07:44 box:shell startMail exec: docker run --restart=always -d --name="mail" --net cloudron --net-alias mail --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:2514 --log-opt syslog-format=rfc5424 --log-opt tag=mail -m 161480704 --memory-swap 536870912 --dns 172.18.0.1 --dns-search=. -e CLOUDRON_MAIL_TOKEN="c90d8342fc12cbde6f76fa607b9099ee15c19c9daff455a37f34f70836ec3df6853fce5727f21806f3aa6800fde3cdd64eaafdd847fcb9e2796bc862c13bab1cabeb899d547951bfa24ef380e011cb83497a8f15b4718a91526585beeba45bafc1905bce45b6f66c89a11813fbf56d6e555167aad8b33d73e063445f29adbe1a" -e CLOUDRON_RELAY_TOKEN="5a263cbc4c816b895aff4f5093759db4d8e1808bc2a778598264647d988536c56b507a43c78462cb73e39803648a549c367daf7196d6abefa4b472755d4ebcfa50164929a5c8a01993a872da54c9186b2fd410d7417634d94d3bb96ec39d682888fd0c356ec89c958ba31200e554e0c270cae5dc12de4826f189ec0d8d98a726" -e LOGLEVEL=info -v "/home/yellowtent/boxdata/mail:/app/data" -v "/home/yellowtent/platformdata/addons/mail:/etc/mail:ro" --label isCloudronManaged=true --read-only -v /run -v /tmp cloudron/mail:3.8.3@sha256:877a1afb99e8cae8c82d5a2fca77840425eb7fafc24360fdd1c9c299e41bcfeb
May 20 10:07:45 box:shell startMail (stdout): f828469c14147c6522aad3f9987df9f9299c577132fcf48656493244f66b2e60
May 20 10:07:45 box:shell startMail (stderr): null
May 20 10:07:45 box:shell stopTurn exec: docker stop turn || true
May 20 10:07:55 box:shell stopTurn (stderr): null
May 20 10:07:55 box:shell stopTurn (stdout): turn
May 20 10:07:55 box:shell removeTurn exec: docker rm -f turn || true
May 20 10:07:55 box:shell removeTurn (stderr): null
May 20 10:07:55 box:shell removeTurn (stdout): turn
May 20 10:07:55 box:shell startTurn exec: docker run --restart=always -d --name="turn" --hostname turn --net host --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:2514 --log-opt syslog-format=rfc5424 --log-opt tag=turn -m 80740352 --memory-swap 268435456 --dns 172.18.0.1 --dns-search=. -e CLOUDRON_TURN_SECRET="a5a72268825e6870fb92af0b8a67d2a" -e CLOUDRON_REALM="my.cloudron.colbywhitaker.me" --label isCloudronManaged=true --read-only -v /tmp -v /run "cloudron/turn:1.5.0@sha256:c59a6da9ea55073ede1ba6329739fca72eddf64c3a3c10280bcc5b7fb8197865"
May 20 10:07:55 box:shell startTurn (stdout): ece770cb799b4faa41ef0ad5b56bfc71dc853f79323df9b76d7dd0dc903678ac
May 20 10:07:55 box:shell startTurn (stderr): null
May 20 10:07:55 box:shell stopMysql exec: docker stop mysql || true
May 20 10:07:58 box:shell removeMysql exec: docker rm -f mysql || true
May 20 10:07:58 box:shell stopMysql (stderr): null
May 20 10:07:58 box:shell stopMysql (stdout): mysql
May 20 10:07:58 box:shell removeMysql (stderr): null
May 20 10:07:58 box:shell removeMysql (stdout): mysql
May 20 10:07:58 box:shell startMysql exec: docker run --restart=always -d --name="mysql" --hostname mysql --net cloudron --net-alias mysql --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:2514 --log-opt syslog-format=rfc5424 --log-opt tag=mysql --dns 172.18.0.1 --dns-search=. --ip 172.18.30.1 -e CLOUDRON_MYSQL_TOKEN=47f6f3699e190f49c1e8d214bb88ba3a0eeccf44c940337e5422da901572815b03a744d81013085d18cd68cd9d4b47b85b2ee29aa51d8e47c5e507d3c60783034f81f48583349ef1d9d2682e14839ec843ef5c07033d61020c9a092c69483cf5e2c45e19d588b86fe9d91da46b91f9ddbd7ba42bee114a8e85955996bd761e1d -e CLOUDRON_MYSQL_ROOT_HOST=172.18.0.1 -e CLOUDRON_MYSQL_ROOT_PASSWORD=bb253eb7337fb564c08f3f94c3273ba0347dea8cc7962c7e50a4534759b949c28d337f54bf598251c54c6714e916da092ade017cba04587c83152154be6dafe55d36115c1ea1aea4ce2d706b16a187feff489a48cd7c45f0a675b849b209ce6822600c315a102144ddf4e77de86e4d076d41d40c5e8515a0fe493e7969d2f3ad -v "/home/yellowtent/platformdata/mysql:/var/lib/mysql" --label isCloudronManaged=true --cap-add SYS_NICE --read-only -v /tmp -v /run "cloudron/mysql:3.3.2@sha256:c19ca4f39ba87fe0c2e99c8bda6c99c1cd1ed3c7a5e331cfe6ec03935675442b"
May 20 10:07:59 box:services Waiting for mysql
May 20 10:07:59 box:shell startMysql (stderr): null
May 20 10:07:59 box:shell startMysql (stdout): 02606b9ba3e9528c83c68f6eb6f7bfbf10bc168b5be922201e3c39af1dda9032
May 20 10:07:59 box:services Attempt 1 failed. Will retry: Network error waiting for mysql: connect ECONNREFUSED 172.18.30.1:3000
May 20 10:08:15 box:services startPostgresql: postgresql will be upgraded
May 20 10:08:15 box:services exportDatabase: Exporting postgresql
May 20 10:08:15 box:shell exportDatabase - removepostgresql exec: docker rm -f postgresql
May 20 10:08:15 box:shell exportDatabase - removepostgresql (stdout): postgresql
May 20 10:08:15 box:shell exportDatabase - removeAddonDirpostgresql spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/rmaddondir.sh postgresql
May 20 10:08:15 box:shell exportDatabase - removepostgresql (stderr): null
May 20 10:08:15 box:shell stopPostgresql exec: docker stop postgresql || true
May 20 10:08:15 box:shell removePostgresql exec: docker rm -f postgresql || true
May 20 10:08:15 box:shell stopPostgresql (stderr): Error response from daemon: No such container: postgresql
May 20 10:08:15 box:shell stopPostgresql (stdout): null
May 20 10:08:15 box:shell removePostgresql (stderr): Error: No such container: postgresql
May 20 10:08:15 box:shell removePostgresql (stdout): null
May 20 10:08:15 box:shell startPostgresql exec: docker run --restart=always -d --name="postgresql" --hostname postgresql --net cloudron --net-alias postgresql --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:2514 --log-opt syslog-format=rfc5424 --log-opt tag=postgresql --dns 172.18.0.1 --dns-search=. --ip 172.18.30.2 --shm-size=128M -e CLOUDRON_POSTGRESQL_ROOT_PASSWORD="c2d2db65b29a8a121152f8cd6c8b10fbd55b7d57316de2dc1cc8492452274f2dd8e84a5428c7a300d57ec1a6de501025db55c0df3b601294a6a817f900414cf9f53af726f9d0d4bc161c08387e05731c33f79af2b15dbd3a2353028f38bd53e533531bb93cda89629c0a7cf9ff0e3d53b56468439f826f7740f8c05239918f50" -e CLOUDRON_POSTGRESQL_TOKEN="6ea90db426b93a60fd356417e68a8fd1d7909204fbc6024979c8f43a14117465cdec14ea996e96a77bf5305e3c54280e7d8ac267e3be31bfd5178f718c5cfa091b149a7d3715ac361998f9ba07670b37d52d44d1cdf00013e77b7b8e1156047e7ba4df2caa06b4ebe93905e181b137217b37919a20792ff84a240a3a550fe3bf" -v "/home/yellowtent/platformdata/postgresql:/var/lib/postgresql" --label isCloudronManaged=true --read-only -v /tmp -v /run "cloudron/postgresql:5.0.1@sha256:51e0676df61aa61f292bf8abbdd7851e351c7580869d46c84f51bd4859644e07"
May 20 10:08:16 box:shell startPostgresql (stderr): null
May 20 10:08:16 box:shell startPostgresql (stdout): fba7a1b1642d084e11e018d7e1cc943300fe815c4817a13ddcdaaf174c3fea3e
May 20 10:08:16 box:services Waiting for postgresql
May 20 10:08:16 box:services Attempt 1 failed. Will retry: Network error waiting for postgresql: connect ECONNREFUSED 172.18.30.2:3000
May 20 10:08:22 box:shell statusUnbound exec: systemctl is-active unbound
May 20 10:08:22 box:shell statusUnbound (stdout): active
May 20 10:08:22 box:shell statusUnbound (stderr): null
May 20 10:08:22 box:shell statusNginx exec: systemctl is-active nginx
May 20 10:08:22 box:shell statusNginx (stderr): null
May 20 10:08:22 box:shell statusNginx (stdout): active
May 20 10:08:30 box:cloudron Getting logs for box
GET /api/v1/courses?enrollment_state=active&per_page=100 404 1.556 ms - 153
GET /api/v1/courses?enrollment_state=active&per_page=100 404 4.770 ms - 153
is up and running. Logs are at /home/yellowtent/platformdata/logs/box.log
May 20 10:08:32 box:services importDatabase: Importing postgresql
May 20 10:08:32 box:shell stopMongodb exec: docker stop mongodb || true
May 20 10:08:35 box:shell stopMongodb (stdout): mongodb
May 20 10:08:35 box:shell stopMongodb (stderr): null
May 20 10:08:35 box:shell removeMongodb exec: docker rm -f mongodb || true
May 20 10:08:35 box:shell removeMongodb (stdout): mongodb
May 20 10:08:35 box:shell removeMongodb (stderr): null
May 20 10:08:35 box:shell startMongodb exec: docker run --restart=always -d --name="mongodb" --hostname mongodb --net cloudron --net-alias mongodb --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:2514 --log-opt syslog-format=rfc5424 --log-opt tag=mongodb --dns 172.18.0.1 --dns-search=. --ip 172.18.30.3 -e CLOUDRON_MONGODB_ROOT_PASSWORD="12ab3f786e7f3a10e58c2756db933457b91fc18f50a71ebc7e1fa77745303d1d4e2d54902cf2bf7c1bb2e70edff35693aa36b513b90f234b33f24219343419cf9c1301abd8932d8d754adbfb9b99588d7eef3cc1a5a43e0297c3da76b271aa2b90e67959c69958639eb7cd0530c149c91c87f491fd14e2234370552065a92663" -e CLOUDRON_MONGODB_TOKEN="0c6cfcb816714746cacd5cd947dd582a9d38dcb90f3fa02179f101dd25305cbe818965faa68f6c26560a849d8d0ddc0059c97d1b2208f3482085451d7fac23d1247376e39ad62f8b1b1d8eeb2923b68fd8dd579b69827067aa6b197ae9ff789fc953704e4473f47b722bd4f8da0802d202fc1a35c212e2d44c381bc173cf33fc" -v "/home/yellowtent/platformdata/mongodb:/var/lib/mongodb" --label isCloudronManaged=true --read-only -v /tmp -v /run "cloudron/mongodb:4.3.1@sha256:6f20108f62d9c41c3325f37253ecb26e8d6a6a03a405a782aa31262e1b64d08b"
May 20 10:08:36 box:shell startMongodb (stdout): cab10f21ecd41220b3256e86b4b949d9c7266626e942de6b986832abee168219
May 20 10:08:36 box:shell startMongodb (stderr): null
May 20 10:08:36 box:services Waiting for mongodb
May 20 10:08:36 box:services Attempt 1 failed. Will retry: Network error waiting for mongodb: connect ECONNREFUSED 172.18.30.3:3000
May 20 10:08:51 box:services Attempt 2 failed. Will retry: Network error waiting for mongodb: connect ECONNREFUSED 172.18.30.3:3000
May 20 10:09:08 box:shell stopGraphite exec: docker stop graphite || true
May 20 10:09:11 box:shell stopGraphite (stdout): graphite
May 20 10:09:11 box:shell stopGraphite (stderr): null
May 20 10:09:11 box:shell removeGraphite exec: docker rm -f graphite || true
May 20 10:09:11 box:shell removeGraphite (stdout): graphite
May 20 10:09:11 box:shell removeGraphite (stderr): null
May 20 10:09:11 box:shell startGraphite exec: docker run --restart=always -d --name="graphite" --hostname graphite --net cloudron --net-alias graphite --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:2514 --log-opt syslog-format=rfc5424 --log-opt tag=graphite -m 80740352 --memory-swap 268435456 --dns 172.18.0.1 --dns-search=. -p 127.0.0.1:2003:2003 -v "/home/yellowtent/platformdata/graphite:/var/lib/graphite" --label isCloudronManaged=true --read-only -v /tmp -v /run "cloudron/graphite:3.3.0@sha256:005addac7e7576f3960b562404ce59442bc861626af0ae0f5122484f5bfcbbc1"
May 20 10:09:12 box:shell startGraphite (stdout): b1aac49f1189980c095f41551d820deaaccaf5f73cc723516c8afb01af9bb57c
May 20 10:09:12 box:shell startGraphite (stderr): null
May 20 10:09:12 box:sftp start: re-creating container
May 20 10:09:12 box:sftp ensureSecrets: generating new sftp keys of type rsa
May 20 10:09:12 box:platform Failed to start services. retry=false (attempt 0): Could not generate sftp rsa keys: Command failed: ssh-keygen -m PEM -t rsa -f "/home/yellowtent/platformdata/sftp/ssh/ssh_host_rsa_key" -q -N ""
May 20 10:09:12 box:cloudron Startup task at index 2 failed: Could not generate sftp rsa keys: Command failed: ssh-keygen -m PEM -t rsa -f "/home/yellowtent/platformdata/sftp/ssh/ssh_host_rsa_key" -q -N "" BoxError: Could not generate sftp rsa keys: Command failed: ssh-keygen -m PEM -t rsa -f "/home/yellowtent/platformdata/sftp/ssh/ssh_host_rsa_key" -q -N ""
at ensureKeys (/home/yellowtent/box/src/sftp.js:36:149)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async start (/home/yellowtent/box/src/sftp.js:60:5)
at async Object.startServices (/home/yellowtent/box/src/services.js:779:9)
at async Object.start (/home/yellowtent/box/src/platform.js:68:13)
at async onActivated (/home/yellowtent/box/src/cloudron.js:82:5)
at async Array.<anonymous> (/home/yellowtent/box/src/cloudron.js:134:9)
May 20 10:10:12 box:shell restartcollectd spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh collectd
May 20 10:10:29 box:updatechecker checkBoxUpdates: checking for updates
May 20 10:10:30 box:updatechecker checkBoxUpdates: Skipping notification of box update 7.4.1 as user was already notified
May 20 10:10:30 box:updatechecker checkAppUpdates: checking for updates
GET /api/v1/courses?enrollment_state=active&per_page=100 404 1.192 ms - 153
May 20 10:12:37 box:locker Lock unreleased platform_start
May 20 10:16:02 box:updatechecker checkBoxUpdates: checking for updates
May 20 10:16:03 box:updatechecker checkBoxUpdates: Skipping notification of box update 7.4.1 as user was already notified
May 20 10:16:03 box:updatechecker checkAppUpdates: checking for updates
POST /api/v1/tasks/1980/stop 409 Conflict task is not active 18.163 ms - 61
POST /api/v1/tasks/1980/stop 409 Conflict task is not active 13.724 ms - 61
GET /api/v1/courses?enrollment_state=active&per_page=100 404 1.074 ms - 153
May 20 10:17:37 box:locker Lock unreleased platform_start

[xdrakon]

I found the final solution in an old post: https://forum.cloudron.io/topic/9053/app-stuck-in-restarting-after-upgrade-7-4-1/2

This seems to have fixed everything. Thank you for the help everyone!

# rm /home/yellowtent/platformdata/sftp/ssh/ssh_host_rsa_key
# rm /home/yellowtent/platformdata/sftp/ssh/ssh_host_rsa_key.pub 
# systemctl restart box