Email for pocketbase

[james]

Hello @valexico
Thanks for reporting this.
I am looking into it.

[james]

This behaviour is very odd.
Sometimes it sends, sometimes it does not, but always the send test email times out.
As if there is an internal event / cron handler that does not work correctly and only if you hit the perfect time frame to send a test mail it actually sends

Many times it fails the send test mail only to send the mail after the failed attempt anyway.
I observed this in the mail service logs and the debug log of pocketbase.

There must be something going in internally that is creating this issue.

[robi]

Could it be hitting throttling limits?

[james]

Hello @robi

The Cloudron mail server was not throttling, I ensured that.

[robi]

@james I was thinking fw not mail.

Next thing would be to see what the code does

[james]

Hello @jesperbllnbm

This is/was a very complex issue to figure out.

TL;DR - "fix" or workaround

Renaming your mail domain from my to e.g.: mail in /#/email-settings will resolve this issue. (be sure to also update your PTR/rDNS record)

Changing the port from 2587 to 587 also works when incoming mails for your Cloudron server is enabled.
Warning, changing just the port in pocketbase will not be permanent, since every app restart will configure 2587 again.

Detailed report

Pocketbase is a golang application.
If CGO_ENABLED=0 is set for the build it will result in Go uses its own built-in resolver.
While designed to mimic the system, it has known edge cases where it might skip /etc/hosts and rely entirely on DNS.

Certain hosting providers, like DigitalOcean, will set the name for the server in the /etc/hosts file.
So if you name your server my.cloudron.dev DigitalOcean will create a record in the /etc/hosts file with my.cloudron.dev 127.0.0.1, which will result into DNS queries asking for my.cloudron.dev to answer with 127.0.0.1.
If this is not handled by Cloudron certain things would not work.
For example, if you'd use the N8N Cloudron app to run API calls against your own Cloudron server, it would resolve my.cloudron.dev to 127.0.0.1 and fail.

This is why we have a line of code to prevent this issue - https://git.cloudron.io/platform/box/-/blob/v9.1.6/src/docker.js?ref_type=tags#L716

if (manifest.id !== 'com.adguard.home.cloudronapp') containerOptions.HostConfig.ExtraHosts.push(`${dashboardFqdn}:172.18.0.1`);

to always resolve the dashboard FQND to 172.18.0.1 which is the docker proxy / docker bridge inside the docker containers, except for the AdGuard app.

But now, something specific can happen.
The golang mail client requires the mail host to match the mail domain for StartTLS validation.
That is why we have the requiresValidCertificate flag for the sendmail addon for Cloudron apps to handle apps that need valid certificates for mailing.
With requiresValidCertificate set to false (the default), the CLOUDRON_MAIL_SMTP_SERVER variable inside the docker containers will be mail which works for most apps.
With CLOUDRON_MAIL_SMTP_SERVER set truethe CLOUDRON_MAIL_SMTP_SERVER will be the mail domain, by default my.DOMAIN.TLD so for me my.cloudron.dev.

For pocketbase, Go uses its own built-in resolver, it now resolves my.cloudron.dev not to 172.18.0.1 but to the public IPv4/IPv6 139.162.188.66/2a01:7e01::2000:3cff:fe7c:9d41 and tries to connect with port 2587 which fails, because port 2587 is only open internally.
So at some point the failing connection times out and Go's will try to resolve my.cloudron.dev not with its internal resolver but from the system, which then sends the mail.
This explains why the test mail is sent after some time, even tho it reports a timeout and fail.

We are looking deeper into this to find a solution that covers these edge cases to prevent such issues in the future.
The roads of DNS can be a harsh journey to navigate.

[james]

We found a solution and this will be part of the next Cloudron release.

[Valexico]

Thank you for your work on this. It was a hard one...
Any idea of the timing of this next release ? Are you speaking of a major release ?

[james]

Hello @valexico

@Valexico said:

Are you speaking of a major release ?

No, the next minor release will include this fix.
Until then, you can either set the port in pocketbase to 587 if incoming fails is enabled on your Cloudron. (not persistent across restarts)
Or you change the mail domain from my to e.g. mail. (persistent across restarts)

[Valexico]

Hi @james
I realised I can't find where to follow the releases of Cloudron. I found this post which does not seem up to date (I am currently on 9.1.6).

Is there a better source of information ?

[james]

Hello @valexico
The changelog of Cloudron can be found here: https://git.cloudron.io/platform/box/-/blob/master/CHANGES

[Valexico]

Hi @james
so the fix will be in the 9.2.0 release ? Do you know when this will be pushed ? I am now on 9.1.6 and cloudron says it's latest ?
I am waiting for this fix to lauch a project to production

[james]

Hello @valexico

@Valexico said:

I am waiting for this fix to lauch a project to production

You don't have to wait for 9.2.0 since I have stated there are other ways to fix this right now.
Go into your mail configuration in the Cloudron dashboard and change the mail server subdomain from my. to mail. and it also solves the issue.

[Valexico]

But this require some DNS update. As the mail server is already used for other app I think waiting for the fix seem more secure

[james]

Hello @Valexico
If the apps are on Cloudron, all should automatically update as well when you change the subdomain for the mail server.
But the RDNS will also need to be updated.
So I can understand why you are hesitant.

[Valexico]

hi @james
New issue here
As a workaround I used a third party SMTP (BREVO)
But when pocketbase restart it is set back to Cloudron SMTP. IS this expected behaviour ? I think this is not a good practice you should allow custom app configuration here ?

https://git.cloudron.io/packages/pocketbase-app/-/blob/main/start.sh?ref_type=heads#L33

This should not be in start.sh it should run only on fresh install and then let the user customize

[james]

Hello @valexico
Some apps we package with optional mail setup configuration that can be disabled in the dashboard.
This app is currently non-optional.
We started to make a difference for apps that only send transactional mail and apps that do not.
PocketBase to our knowledge only sends transactional mails, for what the Cloudron Mail server is perfect.

Can you tell me why you'd like to configure a different mailer for this app specifically?

[Valexico]

@james For 2 reason

• Because at the moment I am still waiting for the Cloudron update allowing to send mails with default configuration (I don't want to mess my working DNS configuration)
• Because for customer facing apps, I think I may have a higher delivery rate with transactional mail services (currently my mails at tagged SPAM on some users)

I think the second reason is valid. But I found a workaround : as this app have a custom domain I can configure an external SMTP for this domain. My only issue currently is the first one (which should be solved soon if I understand)

[Valexico]

@james Even the "sender name" is overwritten on each reboot. I think this is too much as you are supposed to customize the app name here

[Valexico]

I think the update_config should just be moved in the if bloc it would make more sense :