Cloudron Forum

I just went through the migration yesterday. It was mostly copying data from one app running in rescue mode to another running in rescue mode. Its an easy task for an ai agent.

Cloudron version: 10.1.5

Summary

The brute-force rate limit on ports 22, 202 and 222 doesn't only apply to incoming connections. Apps hit it too when they open SSH connections to external servers. Once an app container opens more than 4 new connections to port 22 within 10 seconds, its SYN packets are dropped. Every retry refreshes the recent entry, so an app that keeps retrying never gets out.

Impact

I run Termix (web SSH client / server management) as a custom app. It runs periodic status checks and metrics collection for every configured host. With 6 hosts, two of them reached through a jump host, it opens enough SSH connections per round to hit the limit. After that, every outgoing SSH connection from the Termix container times out, and the app can't reach any server:

Error: Timed out while waiting for handshake
[ERROR] Jump host 1/1 connection failed
[ERROR] Stats collector connection failed

Inside the container, ss -tan shows connections to :22 stuck in SYN-SENT. The same destinations work fine from other app containers and from outside the server.

Steps to reproduce

From any app's web terminal (I used n8n), open a few quick connections to an external port 22 and compare with another port:

for i in 1 2 3 4 5 6 7; do
  timeout 3 bash -c "exec 3<>/dev/tcp/portquiz.net/22" 2>/dev/null && echo "$i open" || echo "$i FAIL"
done
# 1-4 open, 5-7 FAIL

for i in 1 2 3 4 5 6 7; do
  timeout 3 bash -c "exec 3<>/dev/tcp/portquiz.net/2222" 2>/dev/null && echo -n "ok " || echo -n "FAIL "
done
# ok ok ok ok ok ok ok

sleep 15; timeout 3 bash -c "exec 3<>/dev/tcp/portquiz.net/22" && echo open
# open again after 10+ s without new attempts

Destination	Termix container (stays throttled)	Other app containers
github.com:22	timeout	open
portquiz.net:22	timeout	open
portquiz.net:2222	open	open

Cause

In setup/start/cloudron-firewall.sh:

for port in 22 202 222; do
    ipxtables -A CLOUDRON_RATELIMIT -p tcp --dport ${port} -m state --state NEW -m recent --set --name "public-${port}"
    ipxtables -A CLOUDRON_RATELIMIT -p tcp --dport ${port} -m state --state NEW -m recent --update --name "public-${port}" --seconds 10 --hitcount 5 -j CLOUDRON_RATELIMIT_LOG
done
...
ipxtables -I FORWARD 1 -j CLOUDRON_RATELIMIT

CLOUDRON_RATELIMIT is hooked into FORWARD, which makes sense for incoming SSH that gets DNAT'ed to app containers (e.g. Gitea's SSH port). The rules have no source restriction, though, so traffic leaving containers towards the internet also matches. recent keys on the source address, so each app container gets its own counter, shared across all destinations.

Suggested fix

Exclude container-originated traffic from these rules, e.g.:

$iptables -A CLOUDRON_RATELIMIT -p tcp ! -s 172.18.0.0/16 --dport ${port} -m state --state NEW -m recent --set ...
$iptables -A CLOUDRON_RATELIMIT -p tcp ! -s 172.18.0.0/16 --dport ${port} -m state --state NEW -m recent --update ...

(plus the equivalent for the Docker IPv6 subnet in the ip6tables rules, since containers can also go out over IPv6). Alternatively, restrict the match to packets coming in on the external interface.

Current workaround

On the host:

iptables -I CLOUDRON_RATELIMIT 1 -s 172.18.0.0/16 ! -d 172.18.0.0/16 -p tcp --dport 22 -j RETURN

This is lost whenever Cloudron rebuilds its firewall rules, so it isn't a lasting fix.

cloudron-support --troubleshoot

═══ Host ═══
Vendor: netcup Product: KVM Server
Linux: 6.8.0-147-generic
Ubuntu: noble 24.04
Cloudron: 10.1.5
Execution environment: kvm
Processor: AMD EPYC 7702P 64-Core Processor
BIOS pc-i440fx-6.2  CPU @ 2.0GHz x 6
RAM: 16376312KB
Disk: /dev/vda3       256G

═══ System ═══
[OK]	Root disk usage is OK (58%)
[OK]	Memory usage is OK (77%)
[WARN]	Disk I/O on /home/yellowtent (vda, HDD): 180 MB/s write, iowait 30% — sustained iowait at or above 20% means the system is I/O bound. Investigate with 'iotop' or 'pidstat -d 2'.
[OK]	Clock is NTP-synchronized
[OK]	node version is correct
[OK]	IPv6 is enabled in kernel. Public IPv6 address detected
[OK]	docker is running
[OK]	docker version is correct

═══ Networking ═══
[OK]	no incompatible persistent-firewall packages installed
[OK]	DNS is resolving via systemd-resolved
[OK]	unbound is running
[OK]	IPv4 HTTPS to api.cloudron.io/api/v1/helper/public_ip
[OK]	IPv6 HTTPS to api.cloudron.io/api/v1/helper/public_ip
[OK]	IPv4 HTTPS to auth.docker.io/token
[OK]	IPv6 HTTPS to auth.docker.io/token
[OK]	IPv4 HTTPS to acme-v02.api.letsencrypt.org
[OK]	IPv6 HTTPS to acme-v02.api.letsencrypt.org
[OK]	netplan is good

═══ Cloudron Platform ═══
[OK]	MySQL is running
[OK]	nginx is running
[OK]	Dashboard cert is valid (33 days remaining, expires Nov 21 05:12:04 2026 GMT)
[OK]	dashboard is reachable via loopback
[OK]	No pending database migrations
[OK]	Service 'mysql' is running and healthy
[OK]	Service 'postgresql' is running and healthy
[OK]	Service 'mongodb' is lazy-stopped (no installed Cloudron app uses the 'mongodb' addon)
[OK]	Service 'mail' is running and healthy
[OK]	Service 'graphite' is running and healthy
[OK]	Service 'sftp' is running and healthy
[OK]	box v10.1.5 is running
[OK]	Dashboard is reachable via IPv4 (https://my.9wd.eu)
[OK]	Dashboard is reachable via IPv6 (https://my.9wd.eu)
[SKIP]	Domain expiry check — RDAP lookup for 9wd.eu failed (TLD may not support RDAP, or network unreachable)

======== Summary ========
PASS:  31
WARN:  1
FAIL:  0
SKIP:  1

Logs

/home/yellowtent/platformdata/logs/box.log: no errors; the drops happen in the kernel firewall, not in box.

The kernel log shows a drop by CLOUDRON_RATELIMIT_LOG of a SYN going out of an app container (IN=br-… OUT=eth0, SRC = the Termix container, DST = an external SSH server). The log rule is limited to 2 entries/min shared with all the inbound scanner drops, so only a few container drops make it into the log:

Oct 18 17:38:00 kermit kernel: IPTables RateLimit: IN=br-6a947611c07d OUT=eth0 PHYSIN=vethc6c0e7b SRC=172.18.17.61 DST=188.90.223.109 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=14768 DF PROTO=TCP SPT=59040 DPT=22 WINDOW=42340 RES=0x00 SYN URGP=0

/proc/net/xt_recent/public-22 lists container addresses next to the internet scanners (172.18.17.61 is the Termix container):

src=172.18.17.61 ttl: 64 last_seen: 4302953152 oldest_pkt: 7 4302952079, 4302952128, 4302952128, 4302953088, 4302953088, 4302953152, 4302953152, ...

The hook in FORWARD:

-A FORWARD -j CLOUDRON_RATELIMIT
-A CLOUDRON_RATELIMIT -p tcp -m tcp --dport 22 -m state --state NEW -m recent --set --name public-22 ...
-A CLOUDRON_RATELIMIT -p tcp -m tcp --dport 22 -m state --state NEW -m recent --update --seconds 10 --hitcount 5 --name public-22 ...

Yes, i guess it should be phrased the other way around "if your app needs PHP, node or pip use the dedicated base images for these platforms"

I have created an initial version of termix for cloudron: https://ca.cloudron.io/app/eu.9wd.termix

I am not entirely happy with the onboarding flow yet. I decided to go with the header auth from the Cloudron proxy, but termix still wants users to be created before they can login via header auth. So now you get the Cloudron login on first login, get told by termix you need to create first admin user and when you submit this form you get told that password login is disabled because of header auth. Now you need to reload the page and are logged in.

I just noticed that the latest Matrix update also left my install in a "not responding" state (btw. there does not seem to be a notification for this).

Seems to be the same error as codev:

2026-10-15T09:56:22.676Z shell: apptask: /usr/bin/sudo --non-interactive /home/yellowtent/box/src/scripts/configurelogrotate.sh add a19662e3-62af-4022-bf11-f9dabaef48ae /tmp/a19662e3-62af-4022-bf11-f9dabaef48ae.logrotate
2026-10-15T09:56:22.701Z apptask: startApp: starting container
2026-10-15T09:56:22.928Z tasks: updating task 25633 with: {"percent":90,"message":"Configuring reverse proxy"}
2026-10-15T09:56:22.930Z tasks: updating task 25633 with: {"percent":100,"message":"Done"}
2026-10-15T09:56:22.935Z tasks: setCompleted - 25633: {"result":null,"error":null,"percent":100}
2026-10-15T09:56:22.935Z tasks: updating task 25633 with: {"completed":true,"result":null,"error":null,"percent":100}
2026-10-15T09:56:22.936Z taskworker: Task took 42.679 seconds
2026-10-15T09:56:22.937Z Exiting with code 0
2026-10-15T09:56:22Z ==> Ensure we log to console
2026-10-15T09:56:23Z ==> Configuring OIDC auth
2026-10-15T09:56:23Z ==> Configuring synapse
2026-10-15T09:56:23Z ==> Fixing permissions
2026-10-15T09:56:23Z ==> Starting synapse
2026-10-15T09:56:25Z "read_config",
2026-10-15T09:56:25Z )
2026-10-15T09:56:25Z )
2026-10-15T09:56:25Z ^
2026-10-15T09:56:25Z ^
2026-10-15T09:56:25Z ^^^^^^^^^^^^
2026-10-15T09:56:25Z ^^^^^^^^^^^^^^
2026-10-15T09:56:25Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-10-15T09:56:25Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-10-15T09:56:25Z allow_secrets_in_config=allow_secrets_in_config,
2026-10-15T09:56:25Z allow_secrets_in_config=config_args.secrets_in_config,
2026-10-15T09:56:25Z config_dict,
2026-10-15T09:56:25Z synapse.util.check_dependencies.DependencyException: ['beautifulsoup4']
2026-10-15T09:56:25Z check_requirements("url-preview")
2026-10-15T09:56:25Z main()
2026-10-15T09:56:25Z obj.parse_config_dict(
2026-10-15T09:56:25Z self.invoke_all(
2026-10-15T09:56:25Z ~~~~^^
2026-10-15T09:56:25Z ~~~~~~~~~~~~~~~^
2026-10-15T09:56:25Z ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-10-15T09:56:25Z ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
2026-10-15T09:56:25Z ~~~~~~~~~~~~~~~~~~~~~^
2026-10-15T09:56:25Z ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
2026-10-15T09:56:25Z ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-10-15T09:56:25Z "Synapse Homeserver", argv_options
2026-10-15T09:56:25Z )
2026-10-15T09:56:25Z ...<2 lines>...
2026-10-15T09:56:25Z ...<3 lines>...
2026-10-15T09:56:25Z ==> Ensure we log to console
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/app/homeserver.py", line 333, in load_or_generate_config
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/app/homeserver.py", line 478, in main
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/app/homeserver.py", line 505, in <module>
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/config/_base.py", line 438, in invoke_all
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/config/_base.py", line 595, in load_config
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/config/_base.py", line 676, in load_config_with_parser
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/config/_base.py", line 899, in load_or_generate_config
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/config/_base.py", line 919, in parse_config_dict
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/config/repository.py", line 287, in read_config
2026-10-15T09:56:25Z File "/app/code/env/lib/python3.14/site-packages/synapse/util/check_dependencies.py", line 336, in check_requirements
2026-10-15T09:56:25Z File "<frozen runpy>", line 198, in _run_module_as_main
2026-10-15T09:56:25Z File "<frozen runpy>", line 88, in _run_code
2026-10-15T09:56:25Z Synapse 1.163.0 needs beautifulsoup4 for url-preview, but it is not installed
2026-10-15T09:56:25Z Traceback (most recent call last):
2026-10-15T09:56:25Z config = HomeServerConfig.load_or_generate_config(
2026-10-15T09:56:25Z homeserver_config = load_or_generate_config(sys.argv[1:])
2026-10-15T09:56:25Z obj, _ = cls.load_config_with_parser(config_parser, argv_options)
2026-10-15T09:56:25Z raise DependencyException(deps_unfulfilled)
2026-10-15T09:56:25Z res[config_class.section] = getattr(config, func_name)(*args, **kwargs)
2026-10-15T09:56:25Z return cls.load_config(description, argv_options)

With Cloudron 10.1 released i could finally test the healtchecks app I created a few weeks ago: https://ca.cloudron.io/app/healthchecks

The app is integrated into the Cloudron user management via ProxyAuth which required 10.1 for the path exceptions.

Yes, as far as I remember there is currently no exposed ui for this. I think girish posted how to change the manifest somewhere here on the forum, but i cannot find it. It is probably something that needs to be changed in the mysql database of the cloudron instance itself.

Edit: the simple solution is probably to just reinstall the app. There does not seem to be a way to export settings, though.

No, no recent change to this on my side. Once early on I changed the release job to push the json file into a separate branch. Is your installation still using the old location? The new one is https://git.9wd.eu/apps/cloudron-cypht/raw/branch/release/CloudronVersions.json

I think that is already possible. With the last release you can configure an ics source to check free/busy. so as long as the calendar marks off days as busy I think that should work.

I looked into years ago for our old stack but with Cloudron providing its own mail store and smtp stack on top of the read-only nature of the app container it was not really a fit. Nowadays our stack is even more "heavyweight" (due to the need to be able to scale to 10.000s of users on a single node) so I don't think it would be a good fit for an app.

In the simplest case this would be a cron/at command and a preformatted message.

Depending on the use case there may be some other ways. For example in case you want to send out product updates, maybe a newsletter/mailinglist software would be helpful. Here you can schedule mails and also do some tracking. Or if its just personal notifications I have seen that ntfy has scheduled messages that can trigger smtp delivery.

I made a quick smoke test against our ews implementation. Single and recurring meetings seem to be shown at the right place. Its of course a pity that the calendar is not writeable, but I understand that this would also add more complexity.

In regards to the read only calendars I noticed that you still have the delete button on an event in such a calendar. clicking it then gives an error toast notification. It would probably better to signal the read only state already to the event detail screen.

Another thing I noticed (but i dont think its new) is that whenever you trigger a manual sync from the calendar view, the view always jumps to 0:00 once it has completed.

I looked into this a few weeks ago and since Rustdesk does not bring a web ui of its own I switched my investigation to https://github.com/UNITRONIX/BetterDesk instead. That project had its quirks as well (for example it never generated the custom install clients but also did not give me logging to investigate this), so I did not pursue this any further.

But the code and a CloudronVersions.json can be found at https://git.9wd.eu/apps/cloudron-betterdesk for those that want to try it out and possibly continue with it.

Looking forward to try this out.

Hi James, yes thank you for the mention.

Hi @james,

sure that is absolutely no problem. I had claude use the onlyoffice as a base when I made this app, since the onlyoffice app also does not have a license file the euro office app also did not get one it seems.

https://git.9wd.eu/apps/cloudron-euro-office/commit/dbe85f33f5afee650f99e4118f6b11947acb2665

Edit: Some attribution would have been nice https://git.cloudron.io/packages/eurooffice-app

With the v0.1.6 update from this morning the runners that can be deployed to external systems can now be updated from the xyOps dashboard as well.

Some of the plugins from their marketplace need powershell installed, currently debating if I should add this into the app as well.

xyOps is an open source workflow automation system with job scheduling, server monitoring, alerting and ticketing built in. It orchestrates work across a fleet of servers from a single web interface: chain jobs into visual workflows with conditional logic, schedule them with crontab syntax and blackout windows, watch progress live, and get alerted on server or custom metrics.

Jobs are executed by xySat, the lightweight worker agent, which installs on Linux, macOS, Windows and Docker with one command generated from the xyOps interface. Plugins can be written in any language that can speak JSON over STDIO, and the plugin marketplace ships integrations for Slack, Discord, AWS S3, OpenAI and more.

On Cloudron, this package runs the xyOps conductor - web interface, API, scheduler and database - plus, by default, a local xySat worker in the same container, so jobs can run without adding an external server first. Remote workers connect back through the app domain on port 443, so no extra ports need opening.

Features:

• Visual workflow builder with conditional branching
• Cron-style scheduling with blackout windows and job limits
• Live job tracking, output capture and log search
• Server and custom metric monitoring with alerting
• Built-in ticketing tied to jobs and alerts
• Plugin marketplace and a JSON-over-STDIO plugin API
• Secrets, API keys, webhooks and fine grained privileges

xyOps is the successor to Cronicle, by the same author, and is BSD 3-Clause licensed.

Packaging source: https://git.9wd.eu/apps/cloudron-xyops

Store listing: https://ca.cloudron.io/app/xyops

It is a more advanced alternative to Ctfreak which is already available on the Cloudron app store.

Exchange Webservices as as additional calendar type would be nice, but there does not seem to be something in elixir for this.

Yes, I have seen that as well. This comes from the cypht cardav client. Wenn that one is enabled it looks for a server at that address by default. So to get rid of the error either remove the carddav part from CYPHT_MODULES in .env or add the address of a carddav server.

PS: i have removed the module from the default configuration for the next release