Hetzner PTR Record Invalid

[scooke]

I just finished watching Geostorm just now, and I can't help but read these last two posts with a rising panicky voice, as though the future of the world depends on resolving this...

[d19dotca]

FWIW, I have noticed recently that my emails to Gmail addresses in particular are being rate limited and it appears to be because Google sees it as possible spam.

When I ran some tests, I see my DKIM is no longer signed properly even though there's been no DNS changes. I've verified my DNS and everything looks good, but the date this happened according to Google Postmaster Tools is December 19th. I updated Cloudron to 8.2.0 on the night of December 18th (so basically the 19th from any Eastern Time zone or UTC-0 time).

https://unspam.email/results/plVX3ZXGoX shows bad DKIM ("Existing DKIM Signature: The email is not signed with DKIM, whether or not it is a valid signature; Verified DKIM Signature: The email is not signed with a valid DKIM signature.") and a warning for DMARC ("The email "from" domain not matches the DKIM signature "from" domain.").

It's too coincidental with no changes being made to DNS manually that issues start occurring with DKIM as soon as I upgraded Cloudron to 8.2.0, so I am of the mindset something is wrong in Cloudron.

Here's a screenshot of status from Google Postmaster Tools:

Side note: I think the DNS records part about missing PTR record has been fixed now but the Postmaster Tools hasn't updated yet. It turned out this issue exposed a missing PTR on my IPv6 record (but was set correctly for IPv4).

I should also add that Cloudron shows no errors at all when it run the DNS checks, it's all green. Also I ran host commands and see PTR just fine. So I think the issue is more related to DKIM at this point in Cloudron.

% host -t PTR <ip_address>
<ip_address>.in-addr.arpa domain name pointer mail.d19.ca.
% host -t PTR <ipv6_address>  
<ipv6_address>.ip6.arpa domain name pointer mail.d19.ca.

Just FYI, @girish .

[d19dotca]

Not sure if related, but I do see DKIM mentioned as being replaced, and unsure if this is part of the reason or not. Maybe a reach but wanted to share this in case:

Dec 29 21:40:15 70:M 30 Dec 2024 05:40:15.252 * Server initialized
Dec 29 21:40:15 70:M 30 Dec 2024 05:40:15.252 * Ready to accept connections tcp
Dec 29 21:40:15 doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (1300). Counted for protocol services with service_count != 1: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service lmtp { process_limit=100 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 }
Dec 29 21:40:15 doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203). Counted with: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 } + service auth { process_limit=1 }
Dec 29 21:40:15 Warning: service auth { client_limit=1000 } is lower than required under max. load (1300). Counted for protocol services with service_count != 1: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service lmtp { process_limit=100 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 }
Dec 29 21:40:15 Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203). Counted with: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 } + service auth { process_limit=1 }
Dec 29 21:40:15 loaded TLD files:
Dec 29 21:40:15 1=1445
Dec 29 21:40:15 2=8416
Dec 29 21:40:15 3=3642
Dec 29 21:40:15 loaded 9773 Public Suffixes
Dec 29 21:40:15 Mail service endpoint listening on http://:::3000
Dec 29 21:40:15 loglevel: INFO
Dec 29 21:40:15 log format: DEFAULT
Dec 29 21:40:15 Starting up Haraka version 3.0.5
Dec 29 21:40:15 [INFO] [-] [plugins] loading delay_deny
Dec 29 21:40:15 [INFO] [-] [plugins] loading dns-list
Dec 29 21:40:15 [INFO] [-] [plugins] loading helo.checks
Dec 29 21:40:15 [INFO] [-] [plugins] loading headers
Dec 29 21:40:15 [INFO] [-] [plugins] loading tls
Dec 29 21:40:15 [INFO] [-] [core] loading tls.ini
Dec 29 21:40:15 [INFO] [-] [plugins] loading spf
Dec 29 21:40:15 [INFO] [-] [plugins] loading cloudron
Dec 29 21:40:15 [INFO] [-] [plugins] loading rcpt_to.in_host_list
Dec 29 21:40:15 [NOTICE] [-] [plugins] dkim_sign has been replaced by 'dkim'. Please update config/plugins
Dec 29 21:40:15 [INFO] [-] [plugins] loading dkim
Dec 29 21:40:15 [INFO] [-] [plugins] loading spamassassin
Dec 29 21:40:15 [INFO] [-] [plugins] loading queue/smtp_forward
Dec 29 21:40:15 [INFO] [-] [plugins] loading limit
Dec 29 21:40:15 [NOTICE] [-] [server] Listening on [::0]:2525
Dec 29 21:40:15 [INFO] [-] [server] getting SocketOpts for SMTPS server
Dec 29 21:40:15 TypeError: Cannot read properties of undefined (reading 'loopback_is_rejected')
Dec 29 21:40:15 at exports.checkZoneNegative (/app/code/haraka/node_modules/haraka-plugin-dns-list/index.js:347:22)
Dec 29 21:40:15 at exports.check_zone (/app/code/haraka/node_modules/haraka-plugin-dns-list/index.js:372:20)
Dec 29 21:40:15 at async Promise.all (index 0)
Dec 29 21:40:15 at async exports.check_zones (/app/code/haraka/node_modules/haraka-plugin-dns-list/index.js:393:5)
Dec 29 21:40:15 [INFO] [-] [dns-list] will re-test list zones every 30 minutes
Dec 29 21:40:15 [INFO] [-] [server] Creating TLS server on [::0]:2465
Dec 29 21:40:15 [NOTICE] [-] [server] Listening on [::0]:2465
Dec 29 21:40:15 [NOTICE] [-] [server] Listening on [::0]:2587
Dec 29 21:40:15 [INFO] [-] [cloudron] Initializing queue server on port 6000
Dec 29 21:40:15 [INFO] [-] [limit] connected to redis://127.0.0.1:6379/4
Dec 29 21:40:15 [INFO] [-] [outbound/queue] Loading outbound queue from /app/data/haraka-queue
Dec 29 21:40:15 [INFO] [-] [outbound/queue] Loading the queue...
Dec 29 21:40:15 [INFO] [-] [outbound/queue] [pid: undefined] 0 files in my delivery queue
Dec 29 21:40:15 [INFO] [-] [outbound/queue] [pid: undefined] 0 files in my load queue
Dec 29 21:40:15 [INFO] [-] [outbound/queue] [pid: undefined] 2 files in my temp fail queue
Dec 29 21:40:16 INFO [main] 05:40:16,091 org.apache.tika.server.core.TikaServerProcess Starting Apache Tika 3.0.0 server
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: dovecot entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: haraka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: mail-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: solr entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: spamd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: tika entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

[d19dotca]

Last thing to add... here is a screenshot from Google Postmaster tools which shows that the DKIM success rate went down after the upgrade to Cloudron 8.2.0 when taking into account the event dates.

It seems like Cloudron isn't signing the mail with DKIM signatures at all, as if it's been disabled or something. I think we need this patched ASAP, please.

[matix131997]

I also confirm with myself the problem with DKIM and DMARC, which test says that “from” does not match the domain.

I did a test on the site: https://unspam.email/results/uPOw0MP1f2

[matix131997]

I did a comparison between the e-mail that was sent earlier, before version 8.2.0, and now.

Before version 8.2.0

After 8.2.0

[robi]

Can someone test toggling the DKIM setup to see if 8.2 can actually set it?
Or a new sub.domain.. might help narrow it down

[matix131997]

I had a domain that I bought accidentally with the wrong name.

Old Domain:

New Domain:

Half of the settings are gone.

Records in DNS:

[robi]

Did you set up and enable email for the new domain?

EDIT:

[matix131997]

@robi Yes

EDIT: F***, I forgot about that

[matix131997]

I enabled email. I sent email from that domain and still the same as above. And the records are like this after the launch.

[robi]

Records are fine, does the Cloudron status think they're fine?

Does the external test think the records match now?

How are the mail headers?

Let's not miss any testing steps after making changes..

[d19dotca]

The Cloudron status shows everything green in my instance. The DNS records are perfectly fine. The issue is the Haraka SMTP service in Cloudron seems to no longer be signing the messages properly so they are missing DKIM signatures.

[matix131997]

@robi Yes, all points in the status are in green. Message headers the same as above - post #15

[d19dotca]

@nebulon / @girish , I think unfortunately this is coming down to a defect in 8.2.x where the DKIM signatures are not being used to sign outgoing messages.

I'm thinking we will need a patch for that as soon as possible, please as it's having a big impact in connecting to certain mail providers (seems to mostly be Google at the moment but I'm sure others are affected to a degree too). In the meantime, I may need to switch to a different SMTP server / relay service temporarily.

If there's anything I can do to help, please let me know. I can offer SSH connection into my server if you require it too.

[andreasdueren]

@d19dotca said in Hetzner PTR Record Invalid:

@nebulon / @girish , I think unfortunately this is coming down to a defect in 8.2.x where the DKIM signatures are not being used to sign outgoing messages.

I'm thinking we will need a patch for that as soon as possible, please as it's having a big impact in connecting to certain mail providers (seems to mostly be Google at the moment but I'm sure others are affected to a degree too). In the meantime, I may need to switch to a different SMTP server / relay service temporarily.

If there's anything I can do to help, please let me know. I can offer SSH connection into my server if you require it too.

Of THAT could be the case that I'm currently getting lots of failing DKIM reports

[girish]

There is indeed a regression with DKIM signing. This is fixed now. Will get a patch release out asap.

[Kubernetes]

@girish Great, thanks and happy new year !

[d19dotca]

Hi @girish , any chance that the fix could be released today or tomorrow for those of us on 8.2.x?

Also I’m sure you already have this being tracked for the future but I wanted to at least write down my suggestion: I think if possible it may be a great time to add some more automated test cases for the email functionality in order to make sure the DKIM signature exists in messages as an example.

[girish]

@d19dotca 8.2.2 is out now and should fix the crash and the DKIM issue. The crash issue is only mildly tested since upstream has not merged it yet, but maybe you can check if that works fine?