'All green', but I still can't get email sent from cloudron

qwinter

I managed to get cloudron, nodebb running and I'm testing email. PTR record was 'red' but an email to support from racknerd solved it.

These are my settings:

'All green', but I still can't get email sent from cloudron:

What should I try next? Thanks!
Edit:
"Please allow up to 24 hours for this to propagate and reflect globally.

We do not block any ports on our side."
Well, maybe waiting solves everything

Anyone here having experience with transactional email from cloudron/nerdrack? Should I consider a 3rd party?

robi

where are you sending?

qwinter

@robi to a gsuite account.
12 hrs later, still no dice, so it might not be propagation
I'll try another mail provider, titan
EDIT: it works on titan email.
AND... it went to spam on my gsuite account. And on a gmail account.
I market them as 'not spam', and from that point on they still show in my spam on my gsuite and gmail.

This is a bit worrying right?

Is the IP I'm sending from in some kind of google mail blacklist?
Is there anything I can do, or should I go 3rd party for email sending? These are transactional emails from bbnode. I tried a different domain that is hosted with gsuite for email.

So for all purposes, the idea of sending email from this machine is dead in the water?

When marked as 'not spam', it still goes to 'updates' on gmail. So yeah, not a solution

jdaviescoates

@qwinter said in 'All green', but I still can't get email sent from cloudron:

Is the IP I'm sending from in some kind of google mail blacklist?

Sounds like it. And/ or the PTR changes haven't actually propagated yet. Some tips here https://support.google.com/mail/answer/81126

Very common with cheap VPS hosts for many of the IPs to have bad histories. Often possible to warm them up over time though.

There are certainly people on here (e.g. @d19dotca ) who manage to send/ receive loads of email just using their Cloudron. Pretty sure there are other too (there are plenty of circumstances where it is not legal for organisations to use third party email providers).

girish

@qwinter said in 'All green', but I still can't get email sent from cloudron:

When marked as 'not spam', it still goes to 'updates' on gmail. So yeah, not a solution

I think it categorizes things based on the contents of the email. Is this just a simple test mail you are sending out?

Also, a general comment about email is that you find all sorts of problems when using any provider. The general impression is that "it just works" when using other things but this is really not my experience. For most services, I see comments saying "check your spam folder". This comment exists for a reason. In fact, just checking my Update tab now, I see that twitter password reset ended up as 'Update' and there's also a whole bunch of spam there:

I do think a little bit more patience is warranted when self-hosting email. The big providers are opaque and unfriendly (as in, you cannot talk to someone to find out how to resolve problems) but at the same time my opinion is that they are not intentionally trying to sabotage email.

Both cloudron email and my personal email have been self-hosted for over 6 years and we have had no real problems. It's easy to get into a bit of a negative mindset that nothing works when it comes to mail delivery but trust me it works if you are patient with it for a while For example, send email from your server to 5 of your gmail friends and ask them to unspam it and it gets better from there.

scooke

I ran your domain (assuming it's legit) and there are several issues with the SPF records, rDNS (or PTR record - something set up the server), and DKIM record. These are not super hard to fix - just read up on them, run your domain through the sites below to get an idea of the issues, and depending on your domain registrar, allow for up to several hours for any DNS changes to propagate. Once you've settled these, you'll Cloudron is just super for running your email off of.

https://mxtoolbox.com/domain/
https://www.emaillistverify.com/dns-health-checker
https://dnschecker.org/domain-health-checker.php
https://uptime.com/domain-health
https://dmarcian.com/domain-checker/

luckow

It's better to have only one spf record. IMHO it's safe to delete the first one.

dig -t TXT unstuckphd.com                                        

; <<>> DiG 9.18.1 <<>> -t TXT unstuckphd.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6243
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;unstuckphd.com.			IN	TXT

;; ANSWER SECTION:
unstuckphd.com.		300	IN	TXT	"v=spf1 a mx ~all"
unstuckphd.com.		300	IN	TXT	"v=spf1 a:my.unstuckphd.com include:spf.titan.email ~all"

qwinter

@luckow Thanks so much. I got postmark to work. I'm happy something works now so I can move to more important topics Email was not in my mind something that could cause this much trouble.

qwinter

Well, I got too excited too soon. Cloudron/nodebb is not sending email now that I have configured it to use postmarkapp.
From inside nodebb, I see the email sender is town.app@unstuckphd.com. And one cannot change it.

First, is cloudron using the postman relay?

Now when I use 'test email' from cloudron, nothing arrives to gmail addresses. But also NOT to the same domain sending the email (comm@unstuckphd as sender and receiver).

What do we know? Postmark can send email:

curl "https://api.postmarkapp.com/email" \
  -X POST \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -H "X-Postmark-Server-Token: 7651aab8-f197-438e-a1c1-474edc977afc" \
  -d '{
        "From": "comm@unstuckphd.com",
        "To": "comm@unstuckphd.com",
        "Subject": "Hello from Postmark",
        "HtmlBody": "<strong>Hello</strong> dear Postmark user.",
        "MessageStream": "outbound"
      }'

Works:

{"To":"comm@unstuckphd.com","SubmittedAt":"2022-04-19T19:50:27.4884947Z","MessageID":"632b44b6-08ae-4c1c-883e-a834b7fe2f35","ErrorCode":0,"Message":"OK"}⏎

I have no idea what to test next. This 'email not working' is a bottleneck to go live, and it has sucked 2 fulltime days of my life already.

scooke

@qwinter Well, now you need to go change the SPF records since you are using Postmark to send email! I'm pretty sure they will have a page explaining what info you need to include. The reason it won't be sending right now is that the various email systems out there see that the domain is using the Postmark smtp host, without anything about Postmark in the SPF records (which are all at your domain name registrar), but instead either just the domain, or the IP of the Cloudron server, which have nothing to do with Postmark. If you opt for Postmark, they will likely also give instructions on how to set up DKIM and DMARC. Happy reading!!

However, what is receiving your email? If you are using Cloudron to manage your email, it doesn't make sense to use an external SMTP (well, someone here might have a good valid reason, but then they are also likely a DNS Master). So, instead of trying to mix and match the SENDING and the RECEIVING sides of email, just stick with Cloudron, read what you can about SPF, DKIM, DMARC, DNS, starting with the Cloudron documentation, and stick it out. You've already been told that it might take a while for the new domain and email to get "warmed up". And you've been given some tips on how to help that. So, try it. But when you keep mixing and matching... well, you hit a point where it's tough for anyone to definitely say what the problem is.

Email is NOT a mystical black hole, you just need to patiently read the docs.

qwinter

@scooke thanks, I know it can be frustrating to help someone who is trying too many things at the same time and who doesn't want to read the abundant docs everywhere. Email does feel like a black box. Right now there are MX records from titan.email (which I bought from name.com when I got the domain). I can remove those. But then how do I check test emails? I take what you mean is that I should use one of the cloudron webmail apps?

This is what I have on my DNS (name.com) for that domain:

However, what is receiving your email? If you are using Cloudron to manage your email, it doesn't make sense to use an external SMTP

Who is receiving email: gmail accounts for testing (2 gmail, one gsuite) and one titan.email for good measure.
Problem is that when I used cloudron with no relay, email did not arrive to anything but spam. As recommended here, I tried to mark as 'not spam' and reply to these emails, but that didn't solve the problem. It might be I need to do this more and be patient, but this doesn't help: I wanted to have launched this community weeks ago; by going self-hosted I'm adding a few days to the launch. Which is fine, this is a skill I want to learn about self-hosting and even encourage more people to do it.

What I don't want to learn how email works and all the certification layers that need to happen for email to 'just work.' This seems to be a rabbit hole, and an exercise in Yak shaving. If I have to, I will, but I have dozens of things that are higher priority now. Email is smack in the middle of everything. Unless you solve it, you cannot do self-hosting. And it is unpleasant: anyone who doesn't want to have anything to do with sysadmin/servers (99% of the population) will not go through this hoop. I've been on desktop linux for > 20 years and I find it too convoluted. You need to understand way too many things to get it to work. It's like going to a shop to buy a piece of paper, which is just a prerequisite for what you wanted to do, and getting a lecture about growing trees and making paper. A lecture that if you don't attend, you will get no paper. Period.

I'll push it through and get it done. But my hopes of getting anyone around me to get this done themselves have dropped to zero.

This could be a problem: 'test mode, we are reviewing your account'

EDIT: this might explain everything:

Might be waiting 24 hrs 'solves email'. I would still love to know if I should remove the MX entries for titan mail and use one app from cloudron to read email from this domain. The mixed mx entries might be causing trouble.

I think I have set up SPF, DKIM, DMARC with postmark as they instructed. My reason to move to postmark was to avoid getting stuck in spam folder for the time it takes for the domain to warm up. Which coulld be days, and that would kill the project.

luckow

@scooke nope. Emails are received via titan.email

dig -t MX unstuckphd.com                                                                                             

; <<>> DiG 9.18.1 <<>> -t MX unstuckphd.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39675
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;unstuckphd.com.			IN	MX

;; ANSWER SECTION:
unstuckphd.com.		300	IN	MX	20 mx2.titan.email.
unstuckphd.com.		300	IN	MX	10 mx1.titan.email.

scooke

@qwinter Don't feel bad, I wasn't getting on your case for NOT reading the docs, looks like you have been. I'm just trying to encourage you to keep at it.

Although the latest screenshots show yet a different solution?? titan email? titan.email... Ah, I've just googled it. Yeah, I can't help you there atm. I'd need to read their docs! All I can say is choosing Paid-for email services can end up being very tricky, as either there ends up being some artificially enforced limitation that is removable by upgrading, or trying to move away from it ends up being a huge hassle due to, again, a price increase. But I get the draw, the appeal, these services promise of alot of connection and interconnectivity and calendering and scheduling... and you just want to get your community online and running! I wish you all the best!

girish

@qwinter said in 'All green', but I still can't get email sent from cloudron:

From inside nodebb, I see the email sender is town.app@unstuckphd.com. And one cannot change it.

Your configuration of postmark is correct. To change the email address using which an app sends email , please use https://docs.cloudron.io/apps/#mail-from-address . This will automatically configure the app to use the right email address. In addition, nodebb will automatically send via postmark as well.

qwinter

@girish Thanks, it seems all is working now.
Thanks to everyone who saved the day

scooke

@qwinter There were so many details above, can you share what exactly (well, as exact as you can be) what is working? That will be loads of help for the next person looking for similar help.

qwinter

@scooke good point. When I say 'working' I mean simply that emails are getting sent (throug apps in cloudron, in particular nodebb) and they land outside the spam folder.

scooke

@qwinter Yep, I understood that. But, what DNS settings in the end worked?

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

'All green', but I still can't get email sent from cloudron