Clicking on a Domain's email section in the Dashboard causes Cloudron to totally restart!

scooke

So, I noticed my cloudron was seemingly randomly restarting, and I figured it was a RAM thing or something. But all that checks out. Then I noticed one of the domains had a red dot beside it in the Email panel, and when I clicked on the pencil icon, Cloudron restarted again!

The url is like this: https://my.cloudron.com/#/email/example.com

The only relevant logs I can see read:

Jul 10 12:36:57 box:mail Ignored error - dns.dkim: Error: queryTxt ENODATA cloudron-62141b._domainkey.example.com
at QueryReqWrap.onresolve [as oncomplete] (dns.js:203:19) {
errno: undefined,
code: 'ENODATA',
syscall: 'queryTxt',
hostname: 'cloudron-62141b._domainkey.example.com'
Jul 10 12:36:57 box:mail Ignored error - dns.spf: Error: queryTxt ENODATA example.com
at QueryReqWrap.onresolve [as oncomplete] (dns.js:203:19) {
errno: undefined,
code: 'ENODATA',
syscall: 'queryTxt',
hostname: 'example.com'
TypeError: Cannot convert name to ASCII
at Resolver.queryTxt [as resolveTxt] (dns.js:227:48)
at Resolver.resolve (dns.js:263:21)
at Object.resolve (/home/yellowtent/box/src/native-dns.js:32:14)
at checkDkim (/home/yellowtent/box/src/mail.js:220:9)
at /home/yellowtent/box/src/mail.js:480:13
at /home/yellowtent/box/node_modules/async/dist/async.js:2948:28
at eachOfArrayLike (/home/yellowtent/box/node_modules/async/dist/async.js:500:13)
at eachOf (/home/yellowtent/box/node_modules/async/dist/async.js:551:16)
at awaitable (/home/yellowtent/box/node_modules/async/dist/async.js:208:32)
at /home/yellowtent/box/node_modules/async/dist/async.js:2947:9 [ /home/yellowtent/box/box.js:61:17 ]

Something about Cannot convert name to ASCII. The domain name is in English, but it is for a Chinese language site which has no Chinese writing yet.

My domain is set up as Wildcard, and there are only two A records (@, *). The email for this domain is also just Outgoing.

Am I misreading the logs? It sure seems like something about ASCII conversions is messing things up!

EDIT: I managed to sneak my way into the Email settings by following the link to the Outbound settings in the domain's app dashboard (Mail FROM Address
This sets the address from which this app sends email. This app is already configured to send mail using example.com's Outbound Email settings.) and saw that the DKIM and SPF records hadn't been set. Strange, I thought I had. So I set those in the domain registrar. But then it suddenly restarted (showing the same errors as above)!

scooke

Here is an additional twist.

Now I am seeing my Cloudron restart due to trying edit the email settings for an IDN domain (rtl). I don't have an app currently installed.

girish

@scooke When you say "restarting" do you mean the dashboard shows the "cannot connect" red banner on the top? Or do you see some crash somewhere?

The error you are seeing in the logs is not a crash as such, it is just dumping the error. Event for outgoing mail, we have to setup some special DNS records. The dashboard will show the DNS records that you need to setup for outbound mail for the domain.

As for the error, it is saying that the TXT record (SPF) could not be found. Can you check the output of host -t TXT cloudron-62141b._domainkey.example.com on the server (of course, replace it with your domain name).

scooke

@girish By restarting, I mean the logs show me the Cloudron restarted! I'll look again.

scooke

OK, now it seems checking the Email Dashboard for any email makes the Cloudron restart!

Jul 10 22:19:29 box:server ==========================================
Jul 10 22:19:29 box:server Cloudron 6.2.8
Jul 10 22:19:29 box:server ==========================================
Jul 10 22:19:29 box:settings initCache: pre-load settings
Jul 10 22:19:29 box:tasks stopTask: stopping all tasks
Jul 10 22:19:29 box:shell stopTask spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
Jul 10 22:19:29 box:dockerproxy startDockerProxy: started proxy on port 3003
is up and running. Logs are at /home/yellowtent/platformdata/logs/box.log
Jul 10 22:19:29 box:shell removeCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh remove cloudron-backup
Jul 10 22:19:29 box:shell removeCollectdProfile (stdout): Restarting collectd
Jul 10 22:19:29 box:shell removeCollectdProfile (stdout): Removing collectd stats of cloudron-backup
Jul 10 22:19:29 box:reverseproxy writeDashboardConfig: writing admin config for toutdo.com
Jul 10 22:19:30 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
Jul 10 22:19:30 box:platform initializing addon infrastructure
Jul 10 22:19:30 box:platform platform is uptodate at version 48.18.0
Jul 10 22:19:30 box:platform onPlatformReady: platform is ready. infra changed: false
Jul 10 22:19:30 box:apps schedulePendingTasks: scheduling app tasks
Jul 10 22:19:30 box:cron startJobs: starting cron jobs
Jul 10 22:19:31 box:cron backupConfigChanged: schedule 00 00 23 * * 1,3,5 (Europe/Paris)
Jul 10 22:19:31 box:cron autoupdatePatternChanged: pattern - never (Europe/Paris)
Jul 10 22:19:31 box:cron Dynamic DNS setting changed to false
Jul 10 22:19:31 box:notifications alert: id=backupConfig title=Backup configuration is unsafe ack=true
Jul 10 22:19:36 box:cloudron Getting logs for box as json
Jul 10 22:19:42 box:apphealthmonitor app health: 26 alive / 3 dead.

girish

@scooke What are the logs right before the restart logs? There's not obvious crash backtrace?

scooke

@girish
What's below is all in the log. I was on the Domain panel, it had been stable for awhile and I hadn't done anything, but as soon as I clicked on the Email link, it restarted. I wrote anotherdomain.com this time because this was a new domain seemingly connected to this problem. I've even cleared out domains that weren't in use, and rebooted, before this latest restart.

Jul 10 23:06:12 box:apphealthmonitor app health: 26 alive / 3 dead.
Jul 10 23:06:13 box:cloudron Getting logs for box as json
Jul 10 23:06:22 box:apphealthmonitor app health: 26 alive / 3 dead.
TypeError: Cannot convert name to ASCII
at Resolver.queryTxt [as resolveTxt] (dns.js:227:48)
at Resolver.resolve (dns.js:263:21)
at Object.resolve (/home/yellowtent/box/src/native-dns.js:32:14)
at checkDkim (/home/yellowtent/box/src/mail.js:220:9)
at /home/yellowtent/box/src/mail.js:480:13
at /home/yellowtent/box/node_modules/async/dist/async.js:2948:28
at eachOfArrayLike (/home/yellowtent/box/node_modules/async/dist/async.js:500:13)
at eachOf (/home/yellowtent/box/node_modules/async/dist/async.js:551:16)
at awaitable (/home/yellowtent/box/node_modules/async/dist/async.js:208:32)
at /home/yellowtent/box/node_modules/async/dist/async.js:2947:9 [ /home/yellowtent/box/box.js:61:17 ]
Jul 10 23:06:24 box:mail checkRblStatus: anotherdomain.com (ip: 209.182.238.112) servers: []
Jul 10 23:06:29 box:server ==========================================
Jul 10 23:06:29 box:server Cloudron 6.2.8
Jul 10 23:06:29 box:server ==========================================
Jul 10 23:06:29 box:settings initCache: pre-load settings
Jul 10 23:06:29 box:tasks stopTask: stopping all tasks
Jul 10 23:06:29 box:shell stopTask spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
Jul 10 23:06:30 box:dockerproxy startDockerProxy: started proxy on port 3003
is up and running. Logs are at /home/yellowtent/platformdata/logs/box.log
Jul 10 23:06:30 box:shell removeCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh remove cloudron-backup
Jul 10 23:06:30 box:shell removeCollectdProfile (stdout): Restarting collectd
Jul 10 23:06:30 box:shell removeCollectdProfile (stdout): Removing collectd stats of cloudron-backup
Jul 10 23:06:30 box:reverseproxy writeDashboardConfig: writing admin config for toutdo.com
Jul 10 23:06:30 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
Jul 10 23:06:31 box:platform initializing addon infrastructure
Jul 10 23:06:31 box:platform platform is uptodate at version 48.18.0
Jul 10 23:06:31 box:platform onPlatformReady: platform is ready. infra changed: false
Jul 10 23:06:31 box:apps schedulePendingTasks: scheduling app tasks
Jul 10 23:06:31 box:cron startJobs: starting cron jobs
Jul 10 23:06:31 box:cron backupConfigChanged: schedule 00 00 23 * * 1,3,5 (Europe/Paris)
Jul 10 23:06:31 box:cron autoupdatePatternChanged: pattern - never (Europe/Paris)
Jul 10 23:06:31 box:cron Dynamic DNS setting changed to false
Jul 10 23:06:32 box:notifications alert: id=backupConfig title=Backup configuration is unsafe ack=true
Jul 10 23:06:42 box:apphealthmonitor app health: 26 alive / 3 dead.
Jul 10 23:06:42 box:cloudron Getting logs for box as json

girish

@scooke Can you write to us at support@cloudron.io ? Would be good to have this fixed because I can't make out what the issue is. Thanks!

scooke

@girish Ok, will do.

scooke

So, I've sent you an email, but I've been poking around, and removed an IDN domain that didn't have an app installed. Even though it wasn't the initial domain that seemed to spark the problem, it could be related to the other little nugget that was in the logs, something about not being able to translate to ASCII.

If this is related, could there be an issue with IDN domains on Cloudron? I had both a Chinese domain, and a RtL domain. I've removed both, and so far when I go to a few different places in the Dashboard, and then click in Email, everything works smooth, the green circles appear almost instanteously (where before it took awhile).

One of the IDN's didn't actually have an A record pointing to the Cloudron (the Chinese one). Could that have caused some glitch?

nebulon

@scooke right, we are using the native nodejs resolver here https://nodejs.org/dist/latest-v14.x/docs/api/dns.html#dns_class_dns_resolver and it appears to be crashing there.

luckow

@scooke Universal Acceptance is a (not so) big issue for the internet. In short, for domains other than ascii, the internet seems to be broken.
There is an initiative around domains other than ascii called Universal Acceptance https://uasg.tech/
It's not just about Chinese domains. Try sending an email to thomas@müller.de, it will fail.

nebulon

To update, I can reproduce this with a chineses IDN and looking for a fix now.

scooke

@nebulon I wonder why this would pop up now. Both the Chinese domain and the RtL domain were on my Cloudron for a long time. Thanks for looking into it @nebulon.

And interesting link @luckow. I'm reading some of the info there now.

girish

@scooke Could be related to the fact that recent updates upgraded nodejs.

scooke

@luckow Before I send an email to Mr. Thomas Müller, will he be expecting it, or no? Is this someone real?

luckow

@scooke wait. let's see https://müller.de/ is a connected domain (not mine). And maybe there is a thomas there
Send out and say it's only a test

scooke

@luckow The email was returned!

<thomas@xn--mller-kva.de>: host w015e2f2.kasserver.com[85.13.157.81] said: 554
    5.7.1 <thomas@xn--mller-kva.de>: Relay access denied (in reply to RCPT TO command)

However, sending emails to my two IDN domains work fine. Even using the "real" name of the domain and not the punycode rendering. These are though MXroute.

I guess the issue is in the updated nodejs.

nebulon

@scooke we have traced this down to a nodejs bug apparently with the domain in question.
The upstream issue is at https://github.com/nodejs/node/issues/39397

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Solved Clicking on a Domain's email section in the Dashboard causes Cloudron to totally restart!