XMPP Server - Prosody

[djxx]

@nebulon In general, XMPP isn't going to actually serve any web pages. I installed an extra module for health checking because Cloudron requires it - but there's no reason it needs to be exposed externally. BOSH may use HTTP, but it doesn't serve responses that can be interpreted as web pages.

Personally, I don't think it hurts that there's a 404 on an HTTP/S port for an application that doesn't serve HTTP. Putting up a big page like "You found an XMPP server!" just invites trouble from the bad people on the internet.

[nebulon]

Thanks for that input. I think what we really need to add is a way to run apps as headless services. We already have a few of those like game servers and synapse (matrix) which have to serve up a placeholder page. That is not ideal indeed. After Cloudron 9 we will also look into other things like maybe databases, they would have very similar requirements then.

Sorry that this is not yet implemented but we will get there in the future, so it is already great to collect requirements for such service type apps like prosody. This helps to avoid developing the wrong things on the platform side

[robi]

Doesn't Matrix also require something on the root domain? Isn't that where the .well-known endpoint lives?

Something similar can be had?

@djxx, was the architecture chosen to use subdomains vs different ports?

If we're clever with the reverse proxy or with the main app server that listens to connections, it can ferry connections for other services by the request type or layer it needs to access, avoiding the need for extra ports or subdomains.

As for the HTTP page, why not make it multi-useful. For example, by default it's just a Cloudron OIDC login button and then per app it's configured what it does.. for backend services it could be a simple CLI stats page in HTML, or even a web terminal. That way it's flexible and useful to the admin, users, packaging and platform maintainers.

What else would be useful there?

[djxx]

@robi I'm not an expert, but I did do some research on this to respond to Nebulon above. The main thing to note here is that the XMPP protocol expects different domains for different functions. So yes - it's an architectural choice that is mostly beyond our control. Each subdomain represents a different module, and the modules talk to each other (and other servers). They need a way to identify themselves, and they use the domain.

I did some reading and it seems it is theoretically possible to make a server that utilizes different ports to differentiate the modules under a single sub-domain, but this would require tweaking and re-compiling an XMPP server.

Even if this was done, and all XMPP protocols followed, there's always the chance that some not-fully-compliant client that works with every well known XMPP server would not work with ours because we've chosen to deviate so far from the norm, while still technically following the standards. Making these upstream changes would require someone much more familiar with XMPP servers than me.

While I look forward to Cloudron allowing us to package more complex services, I think XMPP is in a pretty good place for the above-average admin; it currently takes less than 5 minutes to set it up. I will probably end up making a cron job to sync the certs into my app's storage volume when I deploy this to production. If I think it's useful to others, and Cloudron 9 isn't out yet, I'll share it here.

[robi]

@djxx Appreciate the response.

We already have some apps that require a secondary domain on the front end, but that isn't necessarily the case for the backend. We just need Nginx to route things as expected by the clients, etc.

Glad you're here, packaging and interested in this stuff working.

[djxx]

@nebulon - as you may see from my other posts, I'm all in on Cloudron ;). I'm in the process of migrating my last server from NethServer to Cloudron; XMPP is one of those services I couldn't move without. I plan to move forward with my manual approach in this thread, but am still interested to see when this could become an official app. Does Cloudron 9 have an ETA?

[nebulon]

@djxx we currently have no ETA for Cloudron 9, but also for this specific case, we haven't implemented anything yet.

I do think eventually we will get there to have better support for non-ui or just backend apps so to speak.

[djxx]

I'm happy to say that I've moved my XMPP server from NethServer to Cloudron. While this is probably not a common move, I am sharing some notes here in case it helps someone else. Also, perhaps this'll cause Cloudron to show up in a few more searches.

• Install XMPP on Cloudron using the steps above. A bit manual for now!

• Dump your ejabberd data (that's the XMPP server NethServer uses) with this command:
  /opt/ejabberd-20.04/bin/ejabberdctl --config-dir /etc/ejabberd dump /etc/ejabberd/xmpp_dump.txt

• Download this dump file locally

• For ease, clone the source for prosody to your local computer so you can utilize the migration tools and not install needless packages on Cloudron. You'll need to run ./configure and ./make - but you don't need to actually install it.

• Don't be a Lua noob. I spent a while struggling to get my Lua environment setup, and thought I needed to run the tools like lua ejabberd2prosody.lua but got lots of errors about dependencies missing. Once I figured out you need to execute it directly like ./ejabberd2prosody.lua things worked fine.

• run the ejabberd2prosody.lua script on your xmpp_dump.txt file:
  ./tools/ejabberd2prosody.lua ~/Desktop/xmpp_migrate/xmpp_dump.txt

• Create a migrator configuration (or use the one I've pasted below). It basically takes everything from the file data format and puts it into the sqlite format, since that's how the Cloudron prosody is configured. Docs:

  • https://prosody.im/doc/migrator
  • https://prosody.im/doc/storage
     

• Run the migrator script:
  ./tools/migration/prosody-migrator.lua --config=./migrator.cfg.lua prosody_files database

• Turn off your Cloudron XMPP app

• Copy the resulting prosody.sqlite file into your Cloudron XMPP's /app/data folder. It will be in the /data folder under your local prosody directory.

• Turn on your Cloudron XMPP app

Your bookmarks, rosters, etc. will now be transferred to your new server! This doesn't appear to move archive messages (mod_mam). Probably because most prosody servers aren't configured to store these permanently so they don't bother migrating them.

I only noticed one issue while migrating. When I first ran the migrator script it gave me errors about topics being empty on some MUCs. I thought I was being smart and edited the code to handle the blanks. This caused me to be unable to join the MUCs on Prosody on certain XMPP clients because Prosody expects there to be a Topic for every MUC.

Once I manually adjusted the MUC topics to be non-empty, the other clients started working fine.

Another almost-issue is that Gajim needed to be restarted a few times to start using OMEMO properly. I think the other MUC issues may have thrown it into an error state.

prosody_files {
    hosts = {
        -- each VirtualHost to be migrated must be represented
        ["domain.com"] = {
            "accounts";
            "account_details";
            "account_flags";
            "account_roles";
            "accounts_cleanup";
            "auth_tokens";
            "invite_token";
            "roster";
            "vcard";
            "vcard_muc";
            "private";
            "blocklist";
            "privacy";
            "archive";
            "archive_cleanup";
            "archive_prefs";
            "muc_log";
            "muc_log_cleanup";
            "persistent";
            "config";
            "state";
            "cloud_notify";
            "cron";
            "offline";
            "pubsub_nodes";
            "pubsub_data";
            "pep";
            "pep_data";
            "skeletons";
            "smacks_h";
            "tombstones";
            "upload_stats";
            "uploads";
        };
        ["conference.domain.com"] = {
            "accounts";
            "account_details";
            "account_flags";
            "account_roles";
            "accounts_cleanup";
            "auth_tokens";
            "invite_token";
            "roster";
            "vcard";
            "vcard_muc";
            "private";
            "blocklist";
            "privacy";
            "archive";
            "archive_cleanup";
            "archive_prefs";
            "muc_log";
            "muc_log_cleanup";
            "persistent";
            "config";
            "state";
            "cloud_notify";
            "cron";
            "offline";
            "pubsub_nodes";
            "pubsub_data";
            "pep";
            "pep_data";
            "skeletons";
            "smacks_h";
            "tombstones";
            "upload_stats";
            "uploads";
        };
    };

    type = "internal"; -- the default file based backend
    path = "/home/user/code/prosody-build/prosody-0.12.4/data/";
}

database {
    -- The migration target does not need 'hosts'
    type = "sql";
    driver = "SQLite3";
    database = "prosody.sqlite";
}

[djxx]

@nebulon - Congrats on the release of version 9! One of my servers just updated and the new UI looks slick. When will we get to see a slick new XMPP app in the store?

[LoudLemur]

Lets support Prosody XMPP server on Cloudron, now that we have Cloudron 9.
https://github.com/prosody/prosody-docker

Docker seems to be the new official way of installing Prosody.

[LoudLemur]

We ran an automated assessment of how difficult it would be to package and then maintain Prosody as an application on Cloudron. The assessment is here (TL/DR - it would be a lot more feasible than Ejabberd):

https://wanderingmonster.dev/blog/monster-manual-prosody/

https://enjoys.rocks/?8957edc0c6a1b7fa#DdSoHsPAZQwxnUHuNVfiQcdFd6soCFf8XLRogrXTNpgy

[LoudLemur]

We managed to deploy Prosody using Cloudron. Here are some notes which we hope might help. For us, Dino was an easier client to use than Kaidan.

Packaging Prosody 13.0.6 (XMPP) for Cloudron: what worked, what bit us

We packaged Prosody 13.0.6 as a Cloudron app at xmpp.example.com with LDAP auth, HTTP file upload, multi-device sync (carbons + MAM), MUC, and 1:1 audio/video via the turn addon. It scores 91% on compliance.conversations.im and passes the connect.xmpp.net TLS/connectivity checks.

The headline finding is a good-news one that contradicts older guidance, so it leads. Then the writeup splits for three audiences: people who just want to run it, people packaging Prosody (or any multi-domain app) for Cloudron, and the Prosody developers.

Built on the shoulders of DerekJarvis/cloudron-prosody (a fork of SaraSmiseth/prosody). Thank you both.

Our packaging (the CloudronManifest, the start script, the cert layout, and the 13.0.6 pin described below) is published at palladium.wanderingmonster.dev/palladium-dragon/prosody-cloudron if you want to reuse or adapt it.

---

TL;DR: the six things worth knowing

1. Cloudron 9.x exposes per-alias TLS certs inside the container, at /etc/certs/<domain>.cert and /etc/certs/<domain>.key, not just the primary tls_cert.pem. This overturns the "primary-domain-only" reading of the tls addon docs and removes the old copy-certs-from-the-host hack for federating component subdomains.
2. Use simple JIDs (user@xmpp.example.com, where the app domain is the VirtualHost). This sidesteps the apex-cert problem and collapses four component subdomains down to one (conference.).
3. LDAP auth means clients must use SASL PLAIN (over TLS). Many clients disable PLAIN by default and then fail in a way that looks exactly like a wrong password.
4. A/V works server-side via the turn addon plus mod_turn_external (XEP-0215). The practical limiter is the client: XMPP A/V calling clients are Linux-desktop only today.
5. A handful of build traps (apt nightly-vs-stable, Podman, registry, core-module conflicts, ENTRYPOINT), with fixes below.
6. The health check needs a real 200. A 404 is treated as unhealthy.

---

(a) For people who just want to run it

JIDs are user@xmpp.example.com. Cloudron users log in with their Cloudron username (or email) as the JID localpart and their Cloudron password. There is nothing to configure inside the XMPP app itself; every Cloudron user is automatically an XMPP user.

One DNS/alias to add: conference.xmpp.example.com (the MUC component, the only thing that federates). Add it as a Cloudron app alias:

cloudron configure --app xmpp.example.com --location xmpp \
  --alias-domains conference.xmpp.example.com

On a Cloudron-managed DNS zone this auto-creates the record and provisions the cert. PEP covers pubsub, file upload is served on the main host, and proxy65 is dropped, so no other subdomains are needed.

Optional SRV records (these improve federation discoverability but are not required, since the JID domain is also the connect host on standard ports):

_xmpp-client._tcp.xmpp.example.com.            300 IN SRV 0 5 5222 xmpp.example.com.
_xmpps-client._tcp.xmpp.example.com.           300 IN SRV 0 5 5223 xmpp.example.com.
_xmpp-server._tcp.xmpp.example.com.            300 IN SRV 0 5 5269 xmpp.example.com.
_xmpp-server._tcp.conference.xmpp.example.com. 300 IN SRV 0 5 5269 xmpp.example.com.

Client login gotcha. If login fails like a wrong password, enable SASL PLAIN (sometimes labelled "allow cleartext auth") in your client. Section (c) explains why this is necessary and why it is safe (c2s requires TLS, so the password only ever travels encrypted).

Encryption. OMEMO is encouraged but not forced (optional policy); c2s requires TLS regardless.

Calls reality check. The server is call-ready, but a working XMPP A/V client is Linux-desktop only right now: Dino, or the experimental calling in Kaidan 0.15. Gajim's A/V is non-functional, and there is no working macOS or mobile XMPP calling client today. Plan for the client side, not the server side.

---

(b) For packagers (Prosody, or any multi-domain Cloudron app)

Base and version

FROM docker.io/cloudron/base:5.0.0 (Ubuntu noble; fully-qualify the name so Podman does not prompt, see the build note below).

Install from the official Prosody apt repo, but pin the stable package:

apt-get install -y prosody=13.0.6-1~noble1

The trap: the prosody-13.0 package is a nightly branch build (it self-reports "13.0 nightly build N"), not the stable point release. Verify with dpkg-query -W prosody. Do not run prosody --version in the build; Prosody's root-guard refuses to run as root and fails the build.

Cloudron specifics

Use CMD, never ENTRYPOINT. ENTRYPOINT breaks Cloudron's debug mode. Put the entrypoint logic in a script invoked by CMD.

Debian FHS paths come with the apt package: config in /etc/prosody, binary /usr/bin/prosody, modules /usr/lib/prosody, Lua 5.4. Point data_path, certificates, run_dir, and pidfile at writable locations (/app/data, /run).

Read addon env on every boot. Never bake CLOUDRON_LDAP_* or CLOUDRON_TURN_* into static config; they change on restart. Map them to your config env in the start script, then gosu prosody:prosody prosody -F.

The health check needs a real 2xx. Cloudron marks the app unhealthy on a 404. We use the community mod_http_host_status_check, which serves GET /host_status_check as HTTP 200, and set healthCheckPath: /host_status_check. Note the distinction: the often-repeated "Prosody 404s in a browser, that's fine" remark applies only to the bare root path a human hits, not to the health path, which must return 200.

Certificates (the headline)

The tls addon exposes the primary cert at /etc/certs/tls_cert.pem and tls_key.pem. On Cloudron 9.x it also exposes a per-alias cert at /etc/certs/<alias-domain>.cert and <alias-domain>.key.

So for a federated MUC subdomain: add it as an alias, then copy /etc/certs/conference.<domain>.{cert,key} into the certs/<domain>/{fullchain,privkey}.pem layout Prosody auto-discovers (chown prosody, key mode 0640). No host-path hack, no cron cert-sync. This is the part that previously forced people into copying the whole host cert directory in, and on 9.x it is no longer necessary.

Wildcard nesting matters: *.example.com covers xmpp.example.com but not conference.xmpp.example.com. The alias yields a *.xmpp.example.com cert, which does.

Modules in 13.0

Many modules older guides copy from the community repo are core now: smacks, turn_external, mam, carbons, csi_simple, muc_mam, server_contact_info, auth_ldap, cloud_notify, and vcard_muc. Copying the community cloud_notify or vcard_muc triggers a "conflict with built-in feature" error; just enable the core ones. We copy only host_status_check, http_host_status_check, e2e_policy, filter_chatstates, and throttle_presence.

Build and deploy

This host runs Podman, not Docker. cloudron build shells out to docker, so bridge it with a docker-to-podman shim early on PATH, plus REGISTRY_AUTH_FILE=~/.docker/config.json so podman push finds Docker's credentials.

cloudron build (local) needs a registry the box can pull from. It builds, pushes, then reads the pushed image's digest for cloudron install. --no-push fails with "Failed to detect sha256". A remote box cannot use a locally-built image without a registry (we used a self-hosted Forgejo container registry). The registry-free "build on the box" experience is the separate Docker Builder app, which still pushes to a registry it manages.

CLI version: there is no 9.x CLI. cloudron tops out at 8.2.6 and works fine against a 9.1.7 box; the CLI and server follow separate version lines.

LDAP

authentication = "ldap"; ldap_mode = "bind"
ldap_server = CLOUDRON_LDAP_SERVER:CLOUDRON_LDAP_PORT
ldap_base   = CLOUDRON_LDAP_USERS_BASE_DN
ldap_rootdn = CLOUDRON_LDAP_BIND_DN ; ldap_password = CLOUDRON_LDAP_BIND_PASSWORD
ldap_filter = "(&(objectclass=user)(|(username=$user)(mail=$user)))"

Cloudron user objects are objectclass=user with username, mail, and uid.

A/V (TURN)

Declare the turn addon. mod_turn_external reads CLOUDRON_TURN_{SERVER,PORT,TLS_PORT,SECRET} and advertises STUN/TURN/TURNS via XEP-0215 with time-limited HMAC REST credentials:

turn_external_host = CLOUDRON_TURN_SERVER   -- the panel host, e.g. my.example.com
turn_external_port = 3478 ; turn_external_tls_port = 5349
turn_external_secret = CLOUDRON_TURN_SECRET

coturn is fronted on the panel host (my.example.com), with a relay UDP range of 50000-51000. Provider-firewall dependency: 3478 and 5349 (TCP+UDP) and the relay range must be reachable from the internet. Cloudron's own firewall opens them; your cloud provider's security group might not. This is the classic "calls connect then drop" cause, so test it before blaming anything else (the WebRTC Trickle ICE page is the quickest check).

Ports / manifest

• httpPort: 5280 (BOSH/websocket/file-upload, fronted by Cloudron TLS on 443).
• tcpPorts: 5222 (c2s STARTTLS), 5223 (c2s direct-TLS, XEP-0368), 5269 (s2s).
• addons: localstorage, tls, ldap, turn; multiDomain: true.

---

(c) For the Prosody developers

Config-sandbox noise. Prosody 13's config sandbox logs a deprecation for every os.getenv/tonumber ("replace os with Lua.os"). For env-driven container configs that is dozens of warning lines per boot. A documented, warning-free idiom for reading env vars in config would help packagers.

SASL and LDAP bind. With bind-mode LDAP, Prosody only offers PLAIN (there is no reusable secret, so no SCRAM). This is correct, but it surprises users whose clients disable PLAIN by default and then see a generic auth failure. A clearer client-facing error ("server offers only PLAIN; enable cleartext-over-TLS") would cut support load. Partly a client issue, but worth a doc note.

Headless/health conventions. The de-facto Cloudron health endpoint (mod_http_host_status_check) lives in community modules. First-class guidance for "Prosody as a backend service behind a managed proxy" (health route, trusted proxy, http_external_url) would help.

Managed coturn. mod_turn_external plus a managed coturn that fronts on a different hostname than the JID works well via XEP-0215 / TURN REST (use-auth-secret). A short doc example would help packagers on Cloudron and other managed platforms.

---

Validation results

compliance.conversations.im: 91% (Prosody 13.0.6 detected). Compliant XEPs include 0215 (STUN+TURN), 0045 (MUC), 0313 (MAM + MUC-MAM), 0280 (Carbons), 0198 (Stream Management), 0363 (HTTP Upload), 0357 (Push), 0384 (OMEMO), 0163 (PEP), 0368, Roster Versioning, 0191 (Blocking), and 0352 (CSI).

TLS. c2s StartTLS (5222), c2s Direct-TLS (5223), and s2s (5269) all present a valid Let's Encrypt certificate. TLS 1.0/1.1 refused, 1.2/1.3 only.

The A/V path. We confirmed the relay path two independent ways: a WebRTC Trickle ICE test gathered a relay candidate from the coturn relay range, and the compliance tester passed XEP-0215 for both STUN and TURN. A live client call additionally confirmed XEP-0353 call signalling routing through Prosody. Worth flagging for anyone testing this: an unanswered call does not by itself exercise TURN, because modern Jingle defers relay allocation until the callee accepts, so a true two-party media test needs two live endpoints.

Federation. The MUC subdomain presents a CA-trusted *.xmpp.example.com certificate on s2s, and Prosody correctly refuses remote servers presenting self-signed certificates.

---

The honest caveat

The server and the Cloudron turn path are call-ready and verified as far as the server's responsibility extends. The practical limiter is XMPP A/V client availability: Dino and Kaidan calling are Linux-desktop only, and there is no working macOS or mobile XMPP calling client today. A live two-party call is the one acceptance item we could not complete, purely for lack of a second Linux endpoint, not because of anything on the server.

---

Credits

DerekJarvis/cloudron-prosody and SaraSmiseth/prosody for the base image and config structure; the Prosody project; and the Cloudron team and the forum threads on the turn and tls addons. Our resulting package is at palladium.wanderingmonster.dev/palladium-dragon/prosody-cloudron.