Email sending broken after updating to 8.2.x (due to IPv6 issues)
-
To summarize the situation:
-
starting 8.2, it seems the mail server has started to prefer using IPv6 for gmail. This wasn't a change in Cloudron consciously at least. I have looked into the Haraka changes and cannot find anything specific there either. I do see that gmail has IPv6 mail servers now, not sure if they were there before or not.
-
To fix the situation, you simply have to set IPv6 PTR record . Cloudron has not implemented a IPv6 PTR check in 8.2 but a check is implemented for next release. The PTR record is set in the VPS provider. Usually, IPv6 is allocated a block of addresses and not a single address like IPv4.
-
If you run
curl https://ipv6.api.cloudron.io/api/v1/helper/public_ip
, this will give you the specific IPv6 address that Cloudron is using to connect to gmail. You have to set the PTR for this specific IPv6 address. -
If your VPS provider does not allow you to set IPv6 PTR , then just disable IPv6 in the interfaces.
sysctl -w net.ipv6.conf.ens18.disable_ipv6=1
for example . You have to putnet.ipv6.conf.ens18.disable_ipv6=1
in your /etc/sysctl.conf for this to persist reboots. After you do this, also disable IPv6 in Cloudron, Network -> IPv6 -> Disable.
wrote on Jan 14, 2025, 4:25 PM last edited by avatar1024 Jan 14, 2025, 5:35 PM@girish said in Email sending broken after updating to 8.2.x (due to IPv6 issues):
To fix the situation, you simply have to set IPv6 PTR record .
For me at least this hasn't worked well. I have three servers. In all three I started to get the issues. I entered a PTR record on all three servers and it checks well with google toolbox https://toolbox.googleapps.com/apps/dig/#PTR/ and with https://www.whatsmydns.net/#PTR.
The IPv6 addresses set for the PTR record are the ones indicated in the email error messages which is the same (I've just checked) than the ones indicated by curl https://ipv6.api.cloudron.io/api/v1/helper/public_ip (and the same detected in the Cloudron Network Setting if IPv6 is activated).
On one of the server I kept getting straight bounce after having set the PTR.
On the two other servers I thought the issue was solved as I wasn't getting bounce anymore but when giving a closer look I saw that some messages would still get errors like:
Delivery failure. Will retry in Xs. Upstream error: 421 4.7.23 [2a03:xxxx:xx:xxx:xxxx:7fff:fe49:51af] The IP address sending this 4.7.23 message does not have a PTR record, or the corresponding forward DNS 4.7.23 entry does not match the sending IP. To protect our users from spam, 4.7.23 mail has been temporarily rate limited. To learn more about IP 4.7.23 address requirements for sending to Gmail, visit 4.7.23 https://support.google.com/a?p=sender-guidelines-ip 4.7.23 To learn more about Gmail requirements for bulk senders, visit 4.7.23 https://support.google.com/a?p=sender-guidelines. 4fb4d7f45d1cf-5d807030e25si25472762a12.537 - gsmtp",Most would get delivered after a couple of tries by the mail server while some would stay in the retry loop indefinitely.
-
-
wrote on Jan 15, 2025, 10:59 AM last edited by
Anyone else experiencing this?
One more thing. On the server where I'm still getting the straight bounce, I've tried to activate IPv6 in the Settings on Cloudron, it worked and the IP is corrected detected. However trying to install apps doesn't work anymore, it stays stuck on Waiting for DNS propagation. So it seems like something is up with the IPv6 set-up on that domain. Any clues on what I need to do? I use wildcard DNS on that domain, do I need to set-up anything manually DNS wise for that domain to work with IPv6?
-
If you use wildcard DNS then you also have to setup the AAAA (ipv6) wildcard DNS record on your own
wrote on Jan 15, 2025, 11:57 AM last edited by avatar1024 Jan 15, 2025, 12:00 PM@nebulon thank you, I thought so but wasn't sure. I've done that and it first glance it seems to have solved both the app install and email bounce issue!
I'll reactivate IPv6 and try those settings on the two other servers and see if all email delivery problem also disappear.
Do I need to also create a AAAA record for the bare domain?
-
-
-
To summarize the situation:
-
starting 8.2, it seems the mail server has started to prefer using IPv6 for gmail. This wasn't a change in Cloudron consciously at least. I have looked into the Haraka changes and cannot find anything specific there either. I do see that gmail has IPv6 mail servers now, not sure if they were there before or not.
-
To fix the situation, you simply have to set IPv6 PTR record . Cloudron has not implemented a IPv6 PTR check in 8.2 but a check is implemented for next release. The PTR record is set in the VPS provider. Usually, IPv6 is allocated a block of addresses and not a single address like IPv4.
-
If you run
curl https://ipv6.api.cloudron.io/api/v1/helper/public_ip
, this will give you the specific IPv6 address that Cloudron is using to connect to gmail. You have to set the PTR for this specific IPv6 address. -
If your VPS provider does not allow you to set IPv6 PTR , then just disable IPv6 in the interfaces.
sysctl -w net.ipv6.conf.ens18.disable_ipv6=1
for example . You have to putnet.ipv6.conf.ens18.disable_ipv6=1
in your /etc/sysctl.conf for this to persist reboots. After you do this, also disable IPv6 in Cloudron, Network -> IPv6 -> Disable.
wrote on Jan 21, 2025, 8:25 PM last edited by@girish said in Email sending broken after updating to 8.2.x (due to IPv6 issues):
To fix the situation, you simply have to set IPv6 PTR record .
And enable IPv6 in Network settings, and then (for good measure) Sync DNS in Domains
-
-
-
-
wrote on Jan 22, 2025, 9:55 PM last edited by
-
Sooo.... Assume that someone doesn't do this every day.
What does one have to do to get email sending to work again?
I'm using netcup and this is the IPV6 screenwrote on Jan 22, 2025, 9:59 PM last edited by jdaviescoates Jan 22, 2025, 10:09 PM@privsec could be wrong, but it doesn't look like there is anywhere for you to and rDNS/ PTR/ reverse dns record for your IPv6. I'd contact netcup support.
ah, seems you should be able to do so but they've just got a crappy unclear UI
https://helpcenter.netcup.com/en/wiki/server/network-server
I'd try whacking your mail url (my
cloudron.domain unless you've changed it) into that empty field with the disk image to the right and then clicking the diskThen check if it worked with
dig -x <your ipv6 address> +short
-
Sooo.... Assume that someone doesn't do this every day.
What does one have to do to get email sending to work again?
I'm using netcup and this is the IPV6 screenwrote on Jan 22, 2025, 10:35 PM last edited by avatar1024 Jan 22, 2025, 10:39 PM@privsec you can do it. In the bottom section (i.e the IPv6 section), enter the full IPv6 address in the field on the left and the PTR record (my.yourdomain.xx) on the right, press save.
You get the full IPv6 address with the command Girish gave above (or by activating IPv6 in Cloudron settings it will show the IPv6 address automatically detected).
-
wrote on Jan 22, 2025, 10:40 PM last edited by
OK, I grabbed the IPv6 addy from cloudron and pasted it in netcup and used the same rDNS name addy for IPv4.
Nwtcup now says to wait 48 hrs
-
OK, I grabbed the IPv6 addy from cloudron and pasted it in netcup and used the same rDNS name addy for IPv4.
Nwtcup now says to wait 48 hrs
wrote on Jan 23, 2025, 8:02 AM last edited by avatar1024 Jan 23, 2025, 9:14 AM@privsec yeah netcup says that but it may only takes a few minutes. You can check your PTR record propagated in various ways, for example:
-
-
wrote on Feb 4, 2025, 1:17 PM last edited by
Got the same issue again, with Outlook servers again. IPv6 is disabled on Cloudron settings and on OS level.
-
Got the same issue again, with Outlook servers again. IPv6 is disabled on Cloudron settings and on OS level.
wrote on Feb 4, 2025, 1:33 PM last edited byfixed by removing IPv6 IP address from Hetzner completely and cleaning up old AAA entries from DNS - they seems like confused Outlook servers.
-
wrote on Feb 6, 2025, 2:49 PM last edited by
I set this up and it worked on netcup for about a week.
Itâs giving me Al the error again about gmails ipv6 not being set up correctly.
Is there an in-depth how to guide to correcting this on netcup?
-
I set this up and it worked on netcup for about a week.
Itâs giving me Al the error again about gmails ipv6 not being set up correctly.
Is there an in-depth how to guide to correcting this on netcup?
wrote on Feb 6, 2025, 4:06 PM last edited by@privsec not netcup specific but the most in depth guide is this post by @avatar1024 :
https://forum.cloudron.io/topic/13072/gmail-ipv6-anyone-else-with-this-experience/22?_=1738857946551
-
wrote on Feb 8, 2025, 11:10 PM last edited by
Also got massive problems sending mails for 2 days now. Possible that 8.2.4 was released that day?
-
Also got massive problems sending mails for 2 days now. Possible that 8.2.4 was released that day?
wrote on Feb 8, 2025, 11:13 PM last edited by@sponch have you sorted out your IPv6 stuff?
-
wrote on Feb 9, 2025, 4:13 AM last edited by sponch Feb 9, 2025, 4:14 AM
Yes. Worked well after doing so for some days. Then âout of the blueâ sending not possible anymore on both of my instances.
âEmail not configured properlyâ errors in notifications when I go to email-overview page it takes 30-40 seconds until the domains get green. All values are set correctly for every single domainâŚ
Log says: Delivery failure, will retry in 65536s.. DNS lookup failure: Error: queryMx ESERVFAIL -
-
wrote on Feb 9, 2025, 9:58 AM last edited by
will try that.
Just found that issue on Hetzner: can that be the reason??
Due to a missing DKIM signature (DomainKey), external mail servers reject your e-mails as spam. For this reason, we have activated DKIM for your domains.If you use our DNS servers for these domains, the DKIM record has been automatically set in the DNS. If you use external DNS servers for these domains, you must also store the displayed DNS record there accordingly. To do this, open the âProductsâ tab, select the domain in question and click on âAdvanced settingsâ under the menu items âE-Mail; DKIM / SPF / DMARCâ.