Hetzner PTR Record Invalid

[jdaviescoates]

@Dave-Swift said in Hetzner PTR Record Invalid:

PTR record is in tact and correct.

@Dave-Swift said in Hetzner PTR Record Invalid:

PTR record began showing as invalid on tools like MXToolbox and WhatsMyDNS. Since then, all outgoing mail has been rejected by recipients, with the failure message indicating that the PTR record is not configured correctly.

I wonder if it's these tools that are failing/ getting it wrong. I note the I seem unable to check DNS or rDNS stuff on Hetzner using MXToolbox recently. Perhaps Hetzner are blocking it at their end or something.

Could be that your mails getting undelivered is unrelated. Are you on Hetzner Cloud? Have you checked if you're on spam blacklists? In the past I've ended up on blacklists because I was in the same range of spammers.

[girish]

@Dave-Swift I was merely removing Cloudron out of the list of suspects

You probably did this, but have you tried host -t PTR <ip> already? For example, host -t PTR 45.55.2.141 returns my.cloudron.io . This helps figuring if it's the tools at fault or if the entry itself is wrong. If entry itself is wrong, only Hetzner can fix this since they own the IP and thus the DNS space for the reverse.

[Dave Swift]

Something weird definitely happened when upgrading to 8.2.1. It looks like it is not the PTR record, but rather DKIM fails (even though the records are correctly added to my DNS).

My bounce messages from Google all specified PTR so that is why I made this post.

I switched to a mailgun relay for the holiday break as I didn't have time to try and troubleshoot this.

DKIM is still out.

As I saw someone else posted in the Freescout, Freescout also stopped being able to check for mail.

[andreasdueren]

@Dave-Swift Is this IPv4 or IPv6 that's failing?

[scooke]

I just finished watching Geostorm just now, and I can't help but read these last two posts with a rising panicky voice, as though the future of the world depends on resolving this...

[d19dotca]

FWIW, I have noticed recently that my emails to Gmail addresses in particular are being rate limited and it appears to be because Google sees it as possible spam.

When I ran some tests, I see my DKIM is no longer signed properly even though there's been no DNS changes. I've verified my DNS and everything looks good, but the date this happened according to Google Postmaster Tools is December 19th. I updated Cloudron to 8.2.0 on the night of December 18th (so basically the 19th from any Eastern Time zone or UTC-0 time).

https://unspam.email/results/plVX3ZXGoX shows bad DKIM ("Existing DKIM Signature: The email is not signed with DKIM, whether or not it is a valid signature; Verified DKIM Signature: The email is not signed with a valid DKIM signature.") and a warning for DMARC ("The email "from" domain not matches the DKIM signature "from" domain.").

It's too coincidental with no changes being made to DNS manually that issues start occurring with DKIM as soon as I upgraded Cloudron to 8.2.0, so I am of the mindset something is wrong in Cloudron.

Here's a screenshot of status from Google Postmaster Tools:

Side note: I think the DNS records part about missing PTR record has been fixed now but the Postmaster Tools hasn't updated yet. It turned out this issue exposed a missing PTR on my IPv6 record (but was set correctly for IPv4).

I should also add that Cloudron shows no errors at all when it run the DNS checks, it's all green. Also I ran host commands and see PTR just fine. So I think the issue is more related to DKIM at this point in Cloudron.

% host -t PTR <ip_address>
<ip_address>.in-addr.arpa domain name pointer mail.d19.ca.
% host -t PTR <ipv6_address>  
<ipv6_address>.ip6.arpa domain name pointer mail.d19.ca.

Just FYI, @girish .

[d19dotca]

Not sure if related, but I do see DKIM mentioned as being replaced, and unsure if this is part of the reason or not. Maybe a reach but wanted to share this in case:

Dec 29 21:40:15 70:M 30 Dec 2024 05:40:15.252 * Server initialized
Dec 29 21:40:15 70:M 30 Dec 2024 05:40:15.252 * Ready to accept connections tcp
Dec 29 21:40:15 doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (1300). Counted for protocol services with service_count != 1: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service lmtp { process_limit=100 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 }
Dec 29 21:40:15 doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203). Counted with: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 } + service auth { process_limit=1 }
Dec 29 21:40:15 Warning: service auth { client_limit=1000 } is lower than required under max. load (1300). Counted for protocol services with service_count != 1: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service lmtp { process_limit=100 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 }
Dec 29 21:40:15 Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203). Counted with: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 } + service auth { process_limit=1 }
Dec 29 21:40:15 loaded TLD files:
Dec 29 21:40:15 1=1445
Dec 29 21:40:15 2=8416
Dec 29 21:40:15 3=3642
Dec 29 21:40:15 loaded 9773 Public Suffixes
Dec 29 21:40:15 Mail service endpoint listening on http://:::3000
Dec 29 21:40:15 loglevel: INFO
Dec 29 21:40:15 log format: DEFAULT
Dec 29 21:40:15 Starting up Haraka version 3.0.5
Dec 29 21:40:15 [INFO] [-] [plugins] loading delay_deny
Dec 29 21:40:15 [INFO] [-] [plugins] loading dns-list
Dec 29 21:40:15 [INFO] [-] [plugins] loading helo.checks
Dec 29 21:40:15 [INFO] [-] [plugins] loading headers
Dec 29 21:40:15 [INFO] [-] [plugins] loading tls
Dec 29 21:40:15 [INFO] [-] [core] loading tls.ini
Dec 29 21:40:15 [INFO] [-] [plugins] loading spf
Dec 29 21:40:15 [INFO] [-] [plugins] loading cloudron
Dec 29 21:40:15 [INFO] [-] [plugins] loading rcpt_to.in_host_list
Dec 29 21:40:15 [NOTICE] [-] [plugins] dkim_sign has been replaced by 'dkim'. Please update config/plugins
Dec 29 21:40:15 [INFO] [-] [plugins] loading dkim
Dec 29 21:40:15 [INFO] [-] [plugins] loading spamassassin
Dec 29 21:40:15 [INFO] [-] [plugins] loading queue/smtp_forward
Dec 29 21:40:15 [INFO] [-] [plugins] loading limit
Dec 29 21:40:15 [NOTICE] [-] [server] Listening on [::0]:2525
Dec 29 21:40:15 [INFO] [-] [server] getting SocketOpts for SMTPS server
Dec 29 21:40:15 TypeError: Cannot read properties of undefined (reading 'loopback_is_rejected')
Dec 29 21:40:15 at exports.checkZoneNegative (/app/code/haraka/node_modules/haraka-plugin-dns-list/index.js:347:22)
Dec 29 21:40:15 at exports.check_zone (/app/code/haraka/node_modules/haraka-plugin-dns-list/index.js:372:20)
Dec 29 21:40:15 at async Promise.all (index 0)
Dec 29 21:40:15 at async exports.check_zones (/app/code/haraka/node_modules/haraka-plugin-dns-list/index.js:393:5)
Dec 29 21:40:15 [INFO] [-] [dns-list] will re-test list zones every 30 minutes
Dec 29 21:40:15 [INFO] [-] [server] Creating TLS server on [::0]:2465
Dec 29 21:40:15 [NOTICE] [-] [server] Listening on [::0]:2465
Dec 29 21:40:15 [NOTICE] [-] [server] Listening on [::0]:2587
Dec 29 21:40:15 [INFO] [-] [cloudron] Initializing queue server on port 6000
Dec 29 21:40:15 [INFO] [-] [limit] connected to redis://127.0.0.1:6379/4
Dec 29 21:40:15 [INFO] [-] [outbound/queue] Loading outbound queue from /app/data/haraka-queue
Dec 29 21:40:15 [INFO] [-] [outbound/queue] Loading the queue...
Dec 29 21:40:15 [INFO] [-] [outbound/queue] [pid: undefined] 0 files in my delivery queue
Dec 29 21:40:15 [INFO] [-] [outbound/queue] [pid: undefined] 0 files in my load queue
Dec 29 21:40:15 [INFO] [-] [outbound/queue] [pid: undefined] 2 files in my temp fail queue
Dec 29 21:40:16 INFO [main] 05:40:16,091 org.apache.tika.server.core.TikaServerProcess Starting Apache Tika 3.0.0 server
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: dovecot entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: haraka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: mail-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: solr entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: spamd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Dec 29 21:40:16 2024-12-30 05:40:16,311 INFO success: tika entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

[d19dotca]

Last thing to add... here is a screenshot from Google Postmaster tools which shows that the DKIM success rate went down after the upgrade to Cloudron 8.2.0 when taking into account the event dates.

It seems like Cloudron isn't signing the mail with DKIM signatures at all, as if it's been disabled or something. I think we need this patched ASAP, please.

[matix131997]

I also confirm with myself the problem with DKIM and DMARC, which test says that “from” does not match the domain.

I did a test on the site: https://unspam.email/results/uPOw0MP1f2

[matix131997]

I did a comparison between the e-mail that was sent earlier, before version 8.2.0, and now.

Before version 8.2.0

After 8.2.0

[robi]

Can someone test toggling the DKIM setup to see if 8.2 can actually set it?
Or a new sub.domain.. might help narrow it down

[matix131997]

I had a domain that I bought accidentally with the wrong name.

Old Domain:

New Domain:

Half of the settings are gone.

Records in DNS:

[robi]

Did you set up and enable email for the new domain?

EDIT:

[matix131997]

@robi Yes

EDIT: F***, I forgot about that

[matix131997]

I enabled email. I sent email from that domain and still the same as above. And the records are like this after the launch.

[robi]

Records are fine, does the Cloudron status think they're fine?

Does the external test think the records match now?

How are the mail headers?

Let's not miss any testing steps after making changes..

[d19dotca]

The Cloudron status shows everything green in my instance. The DNS records are perfectly fine. The issue is the Haraka SMTP service in Cloudron seems to no longer be signing the messages properly so they are missing DKIM signatures.

[matix131997]

@robi Yes, all points in the status are in green. Message headers the same as above - post #15

[d19dotca]

@nebulon / @girish , I think unfortunately this is coming down to a defect in 8.2.x where the DKIM signatures are not being used to sign outgoing messages.

I'm thinking we will need a patch for that as soon as possible, please as it's having a big impact in connecting to certain mail providers (seems to mostly be Google at the moment but I'm sure others are affected to a degree too). In the meantime, I may need to switch to a different SMTP server / relay service temporarily.

If there's anything I can do to help, please let me know. I can offer SSH connection into my server if you require it too.

[andreasdueren]

@d19dotca said in Hetzner PTR Record Invalid:

@nebulon / @girish , I think unfortunately this is coming down to a defect in 8.2.x where the DKIM signatures are not being used to sign outgoing messages.

I'm thinking we will need a patch for that as soon as possible, please as it's having a big impact in connecting to certain mail providers (seems to mostly be Google at the moment but I'm sure others are affected to a degree too). In the meantime, I may need to switch to a different SMTP server / relay service temporarily.

If there's anything I can do to help, please let me know. I can offer SSH connection into my server if you require it too.

Of THAT could be the case that I'm currently getting lots of failing DKIM reports