D
What do the mail service logs show? I cannot send emails out to know this. If you know how to get this let me know. I will post what you need me to on forum.
In Cloudron, you click on Services page and then the log icon for the mail container. Then export the logs (in case the issue isn't most recently seen in the logs).
Now it started working and sending mails. When i started my campaign, I noticed i could not send up to 1000 emails before this issue started again. I am being blocked and my emails will not receive or send. when i try to do a test it does not work and it still keeps showing me the attached. Everything just worked fine when i started the process again but i do not know why after some time i am being blocked....
This definitely sounds like rate limiting being done somewhere if you get up to 1000 emails and then suddenly you can't send any more. But I'm not sure why you can't receive emails either. This feels like something to do with the host you're running with, but if they say they don't do that then I'm not so sure where else it could come from.
What do the mail service logs show? Does the issue happen even after you restart the server or restart the mail service? If yes, then that would seem like something outside of Cloudron, but if the issue is then resolved after restarting then perhaps something else within Cloudron that's affecting things. Best bet to help narrow it down is to show the logs from the mail server (redact sensitive info like email addresses though of course). You may feel safer sending those logs into Cloudron Support email instead from an alternative address.
@girish , I appear to be running into the same issue again. I got an alert that my backup failed to "timedout", and when I checked the Services page it showed one redis instance orange instead of green. Restarting the redis container doesn't seem to do anything to resolve the issue. Things tend to look better when I restart docker entirely.
But even that isn't always enough. The logs show no issues for the redis container. And later on, a different redis container got the same issue. It's very odd. This is definitely a recent issue for me. Seems related to when I installed 8.3.1 but I'm not certain yet.
Logs look fine in the container:
Mar 24 22:16:03 ==> Starting supervisor
Mar 24 22:16:04 2025-03-25 05:16:04,052 INFO Included extra file "/etc/supervisor/conf.d/redis-service.conf" during parsing
Mar 24 22:16:04 2025-03-25 05:16:04,053 INFO Included extra file "/etc/supervisor/conf.d/redis.conf" during parsing
Mar 24 22:16:04 2025-03-25 05:16:04,053 INFO Set uid to user 0 succeeded
Mar 24 22:16:04 2025-03-25 05:16:04,059 INFO RPC interface 'supervisor' initialized
Mar 24 22:16:04 2025-03-25 05:16:04,059 CRIT Server 'inet_http_server' running without any HTTP authentication checking
Mar 24 22:16:04 2025-03-25 05:16:04,059 INFO supervisord started with pid 1
Mar 24 22:16:05 2025-03-25 05:16:05,061 INFO spawned: 'redis' with pid 13
Mar 24 22:16:05 2025-03-25 05:16:05,062 INFO spawned: 'redis-service' with pid 14
Mar 24 22:16:05 13:C 25 Mar 2025 05:16:05.092 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
Mar 24 22:16:05 13:C 25 Mar 2025 05:16:05.099 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
Mar 24 22:16:05 13:C 25 Mar 2025 05:16:05.099 * Redis version=7.4.2, bits=64, commit=00000000, modified=0, pid=13, just started
Mar 24 22:16:05 13:C 25 Mar 2025 05:16:05.099 * Configuration loaded
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.099 * monotonic clock: POSIX clock_gettime
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.105 # Failed to write PID file: Permission denied
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.105 * Running mode=standalone, port=6379.
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.106 * Server initialized
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.106 * Loading RDB produced by version 7.4.2
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.106 * RDB age 169 seconds
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.106 * RDB memory usage when created 3.71 Mb
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.130 * Done loading RDB, keys loaded: 7119, keys expired: 0.
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.130 * DB loaded from disk: 0.024 seconds
Mar 24 22:16:05 13:M 25 Mar 2025 05:16:05.130 * Ready to accept connections tcp
Mar 24 22:16:05 Redis service endpoint listening on http://:::3000
Mar 24 22:16:06 2025-03-25 05:16:06,473 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Mar 24 22:16:06 2025-03-25 05:16:06,474 INFO success: redis-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Mar 24 22:21:06 13:M 25 Mar 2025 05:21:06.032 * 10 changes in 300 seconds. Saving...
Mar 24 22:21:06 13:M 25 Mar 2025 05:21:06.032 * Background saving started by pid 26
Mar 24 22:21:06 26:C 25 Mar 2025 05:21:06.049 * DB saved on disk
Mar 24 22:21:06 26:C 25 Mar 2025 05:21:06.049 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
Mar 24 22:21:06 13:M 25 Mar 2025 05:21:06.132 * Background saving terminated with success
Mar 24 22:26:07 13:M 25 Mar 2025 05:26:07.054 * 10 changes in 300 seconds. Saving...
Mar 24 22:26:07 13:M 25 Mar 2025 05:26:07.055 * Background saving started by pid 27
Mar 24 22:26:07 27:C 25 Mar 2025 05:26:07.073 * DB saved on disk
Mar 24 22:26:07 27:C 25 Mar 2025 05:26:07.073 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
Mar 24 22:26:07 13:M 25 Mar 2025 05:26:07.155 * Background saving terminated with success
Notice that it's listening and ready for connections. However one thing I notice between a container in this state and the green ones is the lack of health check access logs for it. On a working one for example, I see this:
Mar 24 22:16:03 ==> Starting supervisor
Mar 24 22:16:04 2025-03-25 05:16:04,123 INFO Included extra file "/etc/supervisor/conf.d/redis-service.conf" during parsing
Mar 24 22:16:04 2025-03-25 05:16:04,124 INFO Included extra file "/etc/supervisor/conf.d/redis.conf" during parsing
Mar 24 22:16:04 2025-03-25 05:16:04,124 INFO Set uid to user 0 succeeded
Mar 24 22:16:04 2025-03-25 05:16:04,128 INFO RPC interface 'supervisor' initialized
Mar 24 22:16:04 2025-03-25 05:16:04,129 CRIT Server 'inet_http_server' running without any HTTP authentication checking
Mar 24 22:16:04 2025-03-25 05:16:04,129 INFO supervisord started with pid 1
Mar 24 22:16:05 2025-03-25 05:16:05,130 INFO spawned: 'redis' with pid 12
Mar 24 22:16:05 2025-03-25 05:16:05,132 INFO spawned: 'redis-service' with pid 13
Mar 24 22:16:05 12:C 25 Mar 2025 05:16:05.138 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
Mar 24 22:16:05 12:C 25 Mar 2025 05:16:05.138 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
Mar 24 22:16:05 12:C 25 Mar 2025 05:16:05.138 * Redis version=7.4.2, bits=64, commit=00000000, modified=0, pid=12, just started
Mar 24 22:16:05 12:C 25 Mar 2025 05:16:05.138 * Configuration loaded
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.138 * monotonic clock: POSIX clock_gettime
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.139 # Failed to write PID file: Permission denied
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.139 * Running mode=standalone, port=6379.
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.140 * Server initialized
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.140 * Loading RDB produced by version 7.4.2
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.140 * RDB age 170 seconds
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.140 * RDB memory usage when created 10.12 Mb
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.231 * Done loading RDB, keys loaded: 28836, keys expired: 0.
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.231 * DB loaded from disk: 0.091 seconds
Mar 24 22:16:05 12:M 25 Mar 2025 05:16:05.231 * Ready to accept connections tcp
Mar 24 22:16:05 Redis service endpoint listening on http://:::3000
Mar 24 22:16:06 2025-03-25 05:16:06,477 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Mar 24 22:16:06 2025-03-25 05:16:06,481 INFO success: redis-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Mar 24 22:19:39 [GET] /healthcheck
Mar 24 22:21:06 12:M 25 Mar 2025 05:21:06.083 * 10 changes in 300 seconds. Saving...
Mar 24 22:21:06 12:M 25 Mar 2025 05:21:06.083 * Background saving started by pid 30
Mar 24 22:21:06 30:C 25 Mar 2025 05:21:06.148 * DB saved on disk
Mar 24 22:21:06 30:C 25 Mar 2025 05:21:06.149 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
Mar 24 22:21:06 12:M 25 Mar 2025 05:21:06.183 * Background saving terminated with success
Mar 24 22:26:07 12:M 25 Mar 2025 05:26:07.060 * 10 changes in 300 seconds. Saving...
Mar 24 22:26:07 12:M 25 Mar 2025 05:26:07.061 * Background saving started by pid 31
Mar 24 22:26:07 31:C 25 Mar 2025 05:26:07.124 * DB saved on disk
Mar 24 22:26:07 31:C 25 Mar 2025 05:26:07.125 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
Mar 24 22:26:07 12:M 25 Mar 2025 05:26:07.161 * Background saving terminated with success
Here's what it looks like in the UI:
@nebulon Well that is what i do not understand whats going on??
How can I rectify this?
Everything is okay but the mail is not sending or receiving. This has nothing to do with my service provider.
When you say it isn’t sending or receiving… what happens when you try to send email exactly? And for receiving… are the senders getting a bounce back message? If yes, what does the bounce back say?
When you try to send… are you getting an automated error response back? Is your mail client showing a failure to connect to the SMTP server?
We need more information for anyone to help.
Are you certain your DNS records show the correct IP of your server for mail.{your domain.tld} and was that changed recently at all?
Is it possible your IP address is intermittently being blocklisted where it is showing listed and then later on is showing fine, and keeps flip flopping like that perhaps? The blocklist alert in Cloudron has no bearing on your actual mail functionality, it especially won’t have anything to do with receiving mail from others. If on a blocklist it still won’t impact your ability to send unless the receiving end is blocking it due to being on the blocklist, or may be delivered but found in spam boxes instead.
Correct me if I’m wrong as I had looked at storj in the past… but if I recall, I couldn’t choose a region where it’d be stored because it’s sort of a CDN based backup service, so bits of the data are spread around different areas, is that correct though? In other words, I could choose to keep the backups in Canada if I use storj, right?
@nebulon For sure, but it’d still be good to have a value that Dovecot is recommending to be within their guidance / best practice. 
Hopefully that can be corrected in the future.
I guess setting it to 1300 or 1500 for example should suffice and suppress the warning messages.
I am seeing these warnings in the logs when I restarted the mail container in Cloudron. It seems like the client_limit should be increased according to these warnings.
Mar 21 18:21:35 doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (1300). Counted for protocol services with service_count != 1: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service lmtp { process_limit=100 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 }
Mar 21 18:21:35 doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203). Counted with: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 } + service auth { process_limit=1 }
Mar 21 18:21:35 Warning: service auth { client_limit=1000 } is lower than required under max. load (1300). Counted for protocol services with service_count != 1: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service lmtp { process_limit=100 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 }
Mar 21 18:21:35 Warning: service anvil { client_limit=1000 } is lower than required under max. load (1203). Counted with: service managesieve-login { process_limit=100 } + service pop3-login { process_limit=500 } + service imap-urlauth-login { process_limit=100 } + service imap-login { process_limit=500 } + service auth { process_limit=1 }
@girish it definitely didn't look like a memory issue as at the time I found it in the orange state it was showing only about 20% usage of its allocated memory. Mind you I guess it could have been hitting a memory limit earlier and restarted but couldn’t come up for some reason? It was odd as the logs showed it was ready for connections but just nothing happened after that. There weren’t any errors that I could find. I’ll try to review the logs in more detail later on though and attach here if I can.
I suppose it could have been related to the 8.3.1 update as that was installed after the last successful backup, but I am doubtful of that when it was running fine on 8.3.0 earlier and this isn’t related to the PostgreSQL issues found earlier.
Hello,
I got an alert when some backups were failing to run ("request timedout"), but it took me a bit too long for my liking to realize that the issue was not the backup endpoint but rather the fact that a redis container was not in a good state for a particular WordPress app install. Restarting the affected redis container didn't seem to help either, I had to restart Docker entirely for it to resolve itself. Now backups are succeeding, thankfully.
My concern though is that if a service container like redis isn't in a healthy state... shouldn't we be getting alerted to that soon after the event occurs? If not, then I'd suggest this be added. I know we can get alerts for certain events, and I'm realizing that I guess this doesn't apply to services which is unfortunate.
@shrey Interesting, I wonder what the magic number might be for memory then if that was the root cause. It likely depends on each server's environment too, it's probably something like X times the current/average use. My PG usage is typically only running around 512 MB at most, often less than that. So 4 GB was several times more than my daily usage numbers seem to represent.
On a side note with regards to the graphs, in my experience I find the graphs to be not too valuable, I think because it (correct me if I'm wrong though) only updates the usage every 5 minutes, so if it hits its max memory usage immediately in under 5 minutes and restarts for example then it won't necessarily be recorded. I may be wrong, but just my general experience, I tend to view the graphs with a grain of salt.
This is strange to read, as I updated to 8.3.0 over the weekend without any issues (other than a couple of SpamAssassin rules I needed to update because of deprecations in the latest version of SA being used in 8.30 Cloudron). Definitely no outages other than a slightly delayed startup while the databases migrated.
For what it’s worth, I had 4 GB capacity allocated to the PostgreSQL database service, so maybe that helped?
@robi I think the question still becomes… how does one apply that change to the mail container only to send over one IP? The change I made works fine and applies to all containers which is okay overall. But it just doesn’t seem ‘right’, I keep thinking I’m overlooking something.
@joseph Yeah, I tried a few different ways including modifying the Netplan 51-cloud-init.yaml file which is used for adding the additional/floating IP address (this works okay for adding the IP itself), but no matter what I tried (with the help of AI too), nothing allowed Docker to use the floating IP for all outbound communication until it suggested the service to add that rule persistently which seemed to work okay thankfully. It just feels like this is "wrong" somehow though even though it works. I want to believe there's a much easier way to do this.
I'm also surprised nobody else has had this issue, although I suppose most people aren't using floating IPs, and those that do either don't run into the issue or already knew how to work around it which I didn't until today after hours of troubleshooting, haha. Or maybe it's something that Gmail is recently applying in their MTA checks. I'm not sure how I haven't run into this myself before. 
Hello,
I am wanting to make it so that all Cloudron / Docker containers are using a certain outbound IP address as the source instead of a different IPv4 address.
Let's say my IPv4 interface has 1.2.3.4 by default, and I have a floating IP of 6.7.8.9. Right now, it seems like the dedicated server routes traffic through all available IPs at seemingly random. I want to force it to use 6.7.8.9 instead from the Cloudron Docker containers.
Currently, I've been able to do this by creating a system service at /etc/systemd/system/docker-snat.service (for example) to add in a firewall rule. The service looks like this:
[Unit]
Description=Insert SNAT rule for Docker container traffic
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/usr/sbin/iptables -t nat -I POSTROUTING -s 172.18.0.0/16 -o enp3s0f0 -j SNAT --to-source {FLOATING_IP}
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
The context for this request is to do with mail and avoiding issues with FCrDNS (forward-confirmed reverse DNS). Specifically, Gmail started blocking delivery from some emails due to the IP address resolving to my mail server not matching the IP address used to send the message. Cloudron is using the floating IP address when it sets the DNS records, but since that doesn't appear to be the only IP being used for outbound mail, some mail are getting rejected.
Is there a better way to achieve this? I added a floating IP recently to the server and that's what triggered this issue as any mail being sent from the original IPs failed because the DNS records now only contained the floating IP instead for the mail server.
@jdaviescoates said in Cloudron migration to new server: amazing!!!:
@crazybrad said in Cloudron migration to new server: amazing!!!:
pay for DNS queries
I didn't even realise that was a thing. Who charges for such things?
Amazon AWS is one that charges. They charge per-million queries usually. It’s usually really cheap to be fair (like 60 cents per million) but can add up for businesses who generate a ton of queries. Also those typically don’t start until after a free layer of a billion queries or something like that. Haha.
@girish Yes I definitely do use mailing lists, sorry I thought I noted that before. Many (though I don’t think all) of the email directories that are unexpectedly found in this directory are from mailing list addresses. In other words they are on the recipient list of a mailing list.
So it’s as if the mailing list feature is creating these too, but it doesn’t make sense of course to have one on the forwarding addresses. The mailing list email I could see and understand I guess since it’s still trying to identify spam to not forward, but I don’t get why it’s adding the destination addresses in as if they’re mailboxes that it can learn from.
I can’t reproduce on-demand that I’ve seen so far but just giving it time for a day or two and they’ll start to appear. I guess sending an email to the mailing list would trigger it, I can try that too next time I’m at my computer again.
@joseph If that’s true then the filter should be showing both, right?
Right now the labels are definitely mixed up. If the Received filter is meant to show only Saved (for some reason), then that label should be changed from Received to Saved in the filter dropdown. Alternatively if Received is meant to represent both Saved and Received then it should be showing results for both status labels.
Personally I think Received should filter appropriate to only emails with the Received label, and there should perhaps be an additional filter added for Saved to show just Saved messages then too if that’s needed.
How it’s filtering right now definitely doesn’t make any sense. 
Funny enough for seeing this post recently… I’ll be migrating Cloudron to a new dedicated server from another next week as my rental agreement is up soon so it’s time to “upgrade” servers again.
I’ve usually had a pretty solid workflow for the Cloudron migrations, I did a lot of them over the years up until a bit ago, so I’ll be sure to document my experience and return here with any suggestions or tips. Hopefully they can help anyone else who has to do this exercise with an actively used Cloudron server that must keep downtime to an absolute minimum (if not near-zero).
@ccfu The version I'm running appears to be v8.2.4 on Ubuntu 22.04.4 LTS. So I guess this could be something that's already fixed in 8.3.0 then.