Cloudron Forum

avatar1024

@avatar1024 Right so here I had made a mistake, I hadn't disabled IPv6 on the network interface on my server persistently and I reckon it got reactivate after a reboot. I've updated my post here for clarity: https://forum.cloudron.io/post/100505

avatar1024

@joseph said in Forward Email with SES, Got "554 Message rejected: Email address is not verified":

For email forwarding to gmail, there might be a general problem with Gmail+SRS . This hasn't been tested in a while, so we hvae to look into this for next release.

Yes I reckon there is at least one problem with SRS rewrite. I always get a bounce with the forward through Cloudron when the original sender and the final recipient are both Gmail.
FYI, probably also in relation to SRS rewrite I also hit this at some point: https://forum.cloudron.io/post/99711.

Though since the past few days I have noticed other rare occasional bounce just sending from the server to Gmail, and this one two different servers. IPv6 is set-up properly on both, PTR records checks fine, AAAA records are set-up, IPv6 is activated on Cloudron, DNS are synced....but somehow Gmail still fails to see the PTR for the IPv6.

I have disabled IPv6 again but this time persistently, i.e. disabling on Cloudron and copying on the server net.ipv6.conf.eth0.disable_ipv6=1 in /etc/sysctl.conf. So far so good, but something seems definitely up with Gmail and IPv6 with the Cloudron mail server.

avatar1024

Given this only happens with forwards it seems it might have to do with the SRS rewrite. The weird thing is why does it only happens if the original sender is Gmail?

avatar1024

@scooke said in Forward Email with SES, Got "554 Message rejected: Email address is not verified":

advising you to "whitelist forwarder IP addresses" in the SPF field. Have you done that in your DNS? It would like something like: v=spf1 a:mydomain.com ip4:xx.xxx.xxx.xxx ~all

I wonder, does something like that also need to be done for the IPv6 address?

avatar1024

@scooke nope 421 4.7.23 is the Google error code I believe, see https://support.google.com/a/answer/81126?visit_id=638750398959602051-541616874&p=sender-guidelines&rd=1.

The IP(v6) of the server is the one in the error message right after 421 4.7.23 in [] which I've hidden with xxx.

avatar1024

Seems like I might be facing something similar, see my post here: https://forum.cloudron.io/post/101670

I'm using the Cloudron built-in SMTP server.

I wonder if it is also linked to this: https://forum.cloudron.io/post/99711

avatar1024

@jdaviescoates Indeed that looks very similar...thanks I'll post there. In my case I'm using the Cloudron built SMTP server

I wonder if it is also linked to this: https://forum.cloudron.io/post/99711

avatar1024

Ok so I've now noticed that the bounce are only when someone with a Gmail address email say user1@mydomain (mailbox hosted on Cloudron) which then forwards to user1@gmail (redirect through roundcube).

anyuser@mydomain > user1@gmail works (
anyuser@posteo > user1@mydomain > user1@gmail works
anyuser@gmail > user1@mydomain > user1@gmail fails

So something seem to be up with redirect specifically when the sender is a Gmail user and the recipient of the redirect is Gmail user.

Error is the usual IPv6 Google BS:

"Upstream error: 421 4.7.23 [2a03:xxxx:xx:xxx:xxxx:xxxx:xxxx:51af]
The IP address sending this 4.7.23 message does not have a PTR record, or the corresponding forward DNS 4.7.23 entry does not match the sending IP. 
To protect our users from spam, 4.7.23 mail has been temporarily rate limited. To learn more about IP 4.7.23 address requirements for sending to Gmail, visit  4.7.23  https://support.google.com/a?p=sender-guidelines-ip  4.7.23 
To learn more about Gmail requirements for bulk senders, visit 4.7.23  https://support.google.com/a?p=sender-guidelines. a640c23a62f3a-ab7d06caa39si398649666b.93 - gsmtp",  "delay": 128

avatar1024

@joseph said in server seems to be going to sleep and becomes inaccessible:

The browser is local timezone and the server is UTC. You might have to translate the tz .

I'm in the UK so I believe at this time of year UTC = local time?

@joseph said in server seems to be going to sleep and becomes inaccessible:

I recommend uninstall gnome and friends altogether. I have never done this but I expect this won't happen easily since uninstalling a DE is probably not supported . Easiest way is to set up a new server and migrate using cloudron backups.

Yes starting from fresh seems the most sensible...but I really cannot understand what has happened (like I am 100% positive I haven't installed packages manually). I wonder if this was something to do with Netcup. Also since this morning, the server seems kinda stable...a bit worrying though.

Many thanks for your help.

Update: I was able to track the date as to when those packages were installed...and it was when I upgrade to ubuntu 24.04 on 1st Jan 2025. I did that on three other servers without installing a DE so I guess something must have gone wrong when doing this one. But it means the issue I encountered yesterday was somehow due to something else. Maybe with the DNS changes meant the server had somehow network issues and was kept with no activities for a while which sent it to sleep...I don't know.

avatar1024

@jdaviescoates said in Email sending broken after updating to 8.2.x (due to IPv6 issues):

I think the solution is not to disable IPv6 but to fully set it all up.

I'm afraid this is still not working, I keep getting weird intermittent bounce even though it's all set-up...

avatar1024

@jdaviescoates Thank you, much appreciated.

@joseph Many thanks for the reply.

@joseph said in server seems to be going to sleep and becomes inaccessible:

Did you install anything on the server recently?

Nope I didn't install anything manually (say via ssh or otherwise). The only thing I did was to reboot the server from the Cloudron interface to complete updates as per notifications (I did not run the update manually on the server).

@joseph said in server seems to be going to sleep and becomes inaccessible:

Are you saying the server has gnome/xfce/kde?

Yes it looks like it's got Gnome...

Can see it even in the thumbnails from Netcup SCP interface:

VS on another server.

@joseph said in server seems to be going to sleep and becomes inaccessible:

Maybe you can check if those packages are installed?

Seems like there are both Gnome and power management packages installed....which is freaking insane.

root@v2202108132182160313:~# dpkg -l gnome*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                     Version                 Architecture Description
+++-========================================-=======================-============-===========================>
un  gnome-applets                            <none>                  <none>       (no description available)
un  gnome-backgrounds                        <none>                  <none>       (no description available)
un  gnome-bluetooth                          <none>                  <none>       (no description available)
ii  gnome-bluetooth-3-common                 46.0-1ubuntu1           all          GNOME Bluetooth 3 common fi>
ii  gnome-bluetooth-common                   3.34.5-13build3         all          GNOME Bluetooth common files
un  gnome-calendar                           <none>                  <none>       (no description available)
un  gnome-characters                         <none>                  <none>       (no description available)
ii  gnome-control-center                     1:41.7-0ubuntu0.22.04.9 amd64        utilities to configure the >
ii  gnome-control-center-data                1:41.7-0ubuntu0.22.04.9 all          configuration applets for G>
ii  gnome-control-center-faces               1:46.4-0ubuntu0.24.04.1 all          utilities to configure the >
ii  gnome-desktop3-data                      42.9-0ubuntu1           all          Common files for GNOME desk>
un  gnome-documents                          <none>                  <none>       (no description available)
un  gnome-icon-theme                         <none>                  <none>       (no description available)
un  gnome-icon-theme-symbolic                <none>                  <none>       (no description available)
un  gnome-initial-setup                      <none>                  <none>       (no description available)
ii  gnome-keyring                            40.0-3ubuntu3           amd64        GNOME keyring services (dae>
ii  gnome-keyring-pkcs11:amd64               40.0-3ubuntu3           amd64        GNOME keyring module for th>
un  gnome-maps                               <none>                  <none>       (no description available)
ii  gnome-menus                              3.36.0-1.1ubuntu3       amd64        GNOME implementation of the>
ii  gnome-online-accounts                    3.44.0-1ubuntu1         amd64        service to manage online ac>
un  gnome-packagekit                         <none>                  <none>       (no description available)
un  gnome-panel                              <none>                  <none>       (no description available)
ii  gnome-remote-desktop                     42.9-0ubuntu0.22.04.2   amd64        Remote desktop daemon for G>
un  gnome-session                            <none>                  <none>       (no description available)
ii  gnome-session-bin                        42.0-1ubuntu2           amd64        GNOME Session Manager - Min>
ii  gnome-session-common                     42.0-1ubuntu2           all          GNOME Session Manager - com>
un  gnome-session-flashback                  <none>                  <none>       (no description available)
ii  gnome-settings-daemon                    42.1-1ubuntu2.2         amd64        daemon handling the GNOME s>
ii  gnome-settings-daemon-common             42.1-1ubuntu2.2         all          daemon handling the GNOME s>
un  gnome-settings-daemon-schemas            <none>                  <none>       (no description available)
ii  gnome-shell                              42.9-0ubuntu2.2         amd64        graphical shell for the GNO>
ii  gnome-shell                              42.9-0ubuntu2.2         amd64        graphical shell for the GNO>
ii  gnome-shell-common                       42.9-0ubuntu2.2         all          common files for the GNOME >
un  gnome-shell-extension-appindicator       <none>                  <none>       (no description available)
un  gnome-shell-extension-autohidetopbar     <none>                  <none>       (no description available)
un  gnome-shell-extension-caffeine           <none>                  <none>       (no description available)
un  gnome-shell-extension-dash-to-panel      <none>                  <none>       (no description available)
un  gnome-shell-extension-dashtodock         <none>                  <none>       (no description available)
un  gnome-shell-extension-desktop-icons      <none>                  <none>       (no description available)
un  gnome-shell-extension-desktop-icons-ng   <none>                  <none>       (no description available)
un  gnome-shell-extension-multi-monitors     <none>                  <none>       (no description available)
un  gnome-shell-extension-pixelsaver         <none>                  <none>       (no description available)
un  gnome-shell-extension-prefs              <none>                  <none>       (no description available)
un  gnome-shell-extension-taskbar            <none>                  <none>       (no description available)
un  gnome-shell-extension-top-icons-plus     <none>                  <none>       (no description available)
un  gnome-shell-extension-ubuntu-dock        <none>                  <none>       (no description available)
un  gnome-shell-extension-workspaces-to-dock <none>                  <none>       (no description available)
un  gnome-shell-extensions                   <none>                  <none>       (no description available)
un  gnome-shell-pomodoro                     <none>                  <none>       (no description available)
un  gnome-software                           <none>                  <none>       (no description available)
un  gnome-sound-recorder                     <none>                  <none>       (no description available)
ii  gnome-startup-applications               42.0-1ubuntu2           amd64        Startup Applications manage>
un  gnome-terminal                           <none>                  <none>       (no description available)
un  gnome-themes-standard-data               <none>                  <none>       (no description available)
un  gnome-todo                               <none>                  <none>       (no description available)
ii  gnome-user-docs                          46.0-1ubuntu1           all          GNOME Help
un  gnome-user-share                         <none>                  <none>       (no description available)
un  gnome-weather                            <none>                  <none>       (no description available)

root@v2202108132182160313:~# dpkg -l power*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                  Version      Architecture Description
+++-=====================-============-============-===================================================
ii  power-profiles-daemon 0.10.1-3     amd64        Makes power profiles handling available over D-Bus.
ii  powermgmt-base        1.37         all          common utils for power management

@joseph said in server seems to be going to sleep and becomes inaccessible:

This is a DNS resolution error. DNS is not working, test with host api.cloudron.io .

Right so, this seemed to happen as the server wakes up and then it becomes fine (all apps healthcheck fine). I did change the DNS config (i.e. added IPv6 as explained above) so maybe propagation wasn't yet perfect. Anyway I can't see how any of this would be related to the server suddenly a DE

I let the logs open all night and while I can see a pause between 23:37 and 8:30, there are no more DNS errors and the servers has remained up since 8:30. So there are progress. Though in the Cloudron Dashboard it says a backup was completed at 01:10 but I cannot see that in the log...?

Feb 11 23:37:00 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 11 23:37:10 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 11 23:37:20 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 12 08:30:30 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 12 08:30:40 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 12 08:30:50 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive

avatar1024

I started to have the IPv6 email bounce again on one of my server and so I did a bunch of (a priori harmless) things to configure IPv6 properly (I had disabled it all before for the server to use IPv4):

activated IPv6 on the network interface on the server
activated IPv6 on Cloudron (Network > IPv6 > Public IP)
added * and @ AAAA record for the domain
Resync DNS from Cloudron interface

All seemed good. Except that after a few minutes, the dashboard and all apps became inaccessible. I tried to SSH into the server and there it said the connection timed out.

I went on Netcup Server Control Panel, reboot the server and it all came back up right away...but then all went down again about 15min later. I tried to power the server on and off, reboot etc. and every time the same thing happens, it all comes back up right away (all services etc. are fine)...but then goes down again after a few minutes. Then I realised that by going Netcup Server Control Panel in the General tab I could access the server from there instead of SSH. Weird things happened:

From the SCP interface the whole server screen looks black (instead, as in my other server, where you can see it's a terminal asking for a username)
when opening the server, it is in graphical mode! I can see a graphical login screen (instead of the usual terminal one) where I can see my username, enter password and then login into a session
just opening the server into the login screen (without even logging in), then the all server, Cloudron and everything else goes back up.

Any clues??

Below is an example of Cloudron log from the point it's working, then a freeze (18:25), then brought back up (22:23):

Feb 11 18:24:30 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 11 18:24:40 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 11 18:24:50 box:apphealthmonitor app health: 19 running / 3 stopped / 0 unresponsive
Feb 11 22:23:20 box:updatechecker checkAppUpdates: Error getting app update info for 1b0f9d03-8de9-46f0-90c2-06355401006e BoxError: getaddrinfo EAI_AGAIN api.cloudron.io
Feb 11 22:23:20 at Object.getAppUpdate (/home/yellowtent/box/src/appstore.js:267:22)
Feb 11 22:23:20 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
Feb 11 22:23:20 reason: 'Network Error',
Feb 11 22:23:20 details: {},
Feb 11 22:23:20 nestedError: Error: getaddrinfo EAI_AGAIN api.cloudron.io
Feb 11 22:23:20 at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:120:26) {
Feb 11 22:23:20 errno: -3001,
Feb 11 22:23:20 code: 'EAI_AGAIN',
Feb 11 22:23:20 syscall: 'getaddrinfo',
Feb 11 22:23:20 hostname: 'api.cloudron.io',
Feb 11 22:23:20 response: undefined
Feb 11 22:23:20 }
Feb 11 22:23:20 }
Feb 11 22:23:20 box:shell Running as unit: box-task-18590.service

Update: another thing that I did earlier that day (even before the whole IPv6 stuff was to reboot the server from the Cloudron notifications because of updates...maybe the issue started then, but I don't know as I went on to do the IPv6 changes after the reboot.

avatar1024

I've started to have this issue again randomly (emails only sometimes bounce...helpful I know) despite having IPv6 is disabled on both Cloudron and on the Network interface for that server.

avatar1024

@timconsidine said in PTR / Gmail:

I don’t use IPV6 but will check it out

Even if you don't have IPv6 activated.on Cloudron but that your server network interface has IPv6 activated, then Google uses IPv6 for emails...it's confused a hell of a lot of us

avatar1024

@girish But syncing Cloudron group might be super useful in other cases, at least that is certainly the case with the organisation I work with and it was a relief when it started to work with LDAP.

How it should be implemented I'm not sure. I understand the concerns of @jdaviescoates when the way app access work is that, apps have access to everything unless you restrict it and give access to only specific users/groups. So following that logic it seems like the behaviour of all groups being synced (as described by @jdaviescoates) is normal (as it does for users) and that if an app should only see some groups/users, then the Cloudron admin should make sure only those groups are granted access (and then the OIDC plugin should only sync those groups and not others).

The Nextcloud admin group however should be independent of OIDC syncing and Nextcloud admins should be able to manage it independently.

avatar1024

Right so we have a bunch of similar topics referencing the same problem.

Should they all be merged and/or marked as solved?

The solution is provided by @girish (here) and @jdaviescoates (here) which I'll compile and summarise here again:

Activate IPv6 on Cloudron via going to Network > IPv6 > Configure > Public IP
Check your IPv6 address either via reading the IPv6 address detected by Cloudron when doing 1. or via running curl https://ipv6.api.cloudron.io/api/v1/helper/public_ip on your server (via ssh).
Set an IPv6 PTR record on your VPS/server provider (not your domain provider) for the above IPv6 address. The next Cloudron release will implement a check on IPv6 PTR record like it currently does for IPv4.
If using Wildcard DNS then create a * AAAA record for the above IPv6 address.
If things still don't work, you can go to Cloudron -> Domains -> hit Sync DNS

If your VPS provider does not allow you to set IPv6 PTR, then disable IPv6 that way:

disable IPv6 in Cloudron (Network -> IPv6 -> Disable)
on your server via ssh run sysctl -w net.ipv6.conf.ens18.disable_ipv6=1 (replacing ens18 for your specific network interface, in my case eth0)
make it persistent by adding net.ipv6.conf.ens18.disable_ipv6=1 to /etc/sysctl.conf (replacing ens18 for your specific network interface, in my case eth0)

Update: I personally still faced issues with Gmail using IPv6 so I ended up disabling it persistently.

avatar1024

@privsec yeah netcup says that but it may only takes a few minutes. You can check your PTR record propagated in various ways, for example:

avatar1024

@privsec you can do it. In the bottom section (i.e the IPv6 section), enter the full IPv6 address in the field on the left and the PTR record (my.yourdomain.xx) on the right, press save.

You get the full IPv6 address with the command Girish gave above (or by activating IPv6 in Cloudron settings it will show the IPv6 address automatically detected).

avatar1024

@nebulon thank you, I thought so but wasn't sure. I've done that and it first glance it seems to have solved both the app install and email bounce issue!

I'll reactivate IPv6 and try those settings on the two other servers and see if all email delivery problem also disappear.

Do I need to also create a AAAA record for the bare domain?

avatar1024

Anyone else experiencing this?

One more thing. On the server where I'm still getting the straight bounce, I've tried to activate IPv6 in the Settings on Cloudron, it worked and the IP is corrected detected. However trying to install apps doesn't work anymore, it stays stuck on Waiting for DNS propagation. So it seems like something is up with the IPv6 set-up on that domain. Any clues on what I need to do? I use wildcard DNS on that domain, do I need to set-up anything manually DNS wise for that domain to work with IPv6?

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Cloudron Forum

avatar1024

Posts