XMPP Server - Prosody
-
I took the time to take a bit closer into the package. There is a long way to go to make it compliant with Cloudron packaging. It is a great start and we don't provide good information on what is expected from Cloudron packages, so this is mostly our fault.
For a start, you already added the
tlsaddon, which does provide the certs in/etc/certs/using that instead of the extra volume mount will ensure the app gets restarted and will pick up the correct certs if they get renewed. I am playing a bit from your package to hopefully get this working.One question is, apparently certs for the root domain are also required, even though no DNS records for that are? This can be solved by also adding the root domain as an alias to the app.
For the extra required DNS records, we have to see if and how we can integrate this in the platform to support those. We already have some well-known type records, maybe it fits there.
Otherwise there are naturally discrepancies elsewhere, given that this started (as far as I can tell) from some upstream Dockerimage which will not fit, but we can get through one by one. The first I noticed was, that currently it doesn't work with the Cloudron app debug mode, since it uses
ENTRYPOINTin Dockerfile, that was easy to change by relying only onCMD@nebulon Thanks for taking a look through. I am certainly open to making changes to make it a compliant package - I just don't know what those changes are.
I didn't realize that the TLD could be added as an alias which would give the cert. I was using /etc/certs before I did the hacky workaround to get the TLD. I will try the approach you mentioned and make sure things are still working.
One question is, apparently certs for the root domain are also required, even though no DNS records for that are?
Correct. The XMPP specification uses the SRV records to point to a specific server. It does need the cert to validate usernames like
user@domain.comeven if you chose to have the XMPP server somewhere else completely.Otherwise there are naturally discrepancies elsewhere, given that this started (as far as I can tell) from some upstream Dockerimage which will not fit, but we can get through one by one.
Can you tell me a bit more about this? I changed it to use the Cloudron base, and confirmed that all the actions it does to build the server are successful. I did my best to follow putting data under /app/data, and runtime files under /run, according to the documentation.
For the extra required DNS records, we have to see if and how we can integrate this in the platform to support those. We already have some well-known type records, maybe it fits there.
This would be nice, but it's really not that difficult (compared to the entirety of setting up an XMPP server). I think if the package was installable and the only extra steps are some DNS changes, it'd still be very easy for people who want XMPP to do.
-
Yes, we have to add this to the platform as a feature as well as think about how to make the DNS setup for those extra domains required. We have a policy that apps should "work" after installation. Do you know if there is a way that maybe those extra (sub)domains and thus their certs can be optional, just limiting the functionality of the app?
-
The root cert will always be needed to get
user@domain.comaddresses, and not having the other sub-domains would make the app basically unusable. The extra sub-domains are for things like: uploading files, proxying connections when a direct connection is unavailable, and for allowing people from other serveres to contact users on your XMPP instance. So, if all the extra domains were removed you would only have an XMPP chat that works locally on your server, and without pictures. These limitations defeat the person of having XMPP, in my opinion.After looking again, it seems we could drop the "proxy" sub-domain since we're using Cloudron's TURN server (I would need to test). But there's still a non-zero number of extra domains needed, as well as the TLD cert.
-
Agree that those are required for a real instance, I was just trying to understand if we can provide a package where the app does not crash on initial startup until those are set up.
All in all since we at least need changes to the TLS addon to also provide the root domain certs, we are looking at least to a package release after the next Cloudron version only. Just to keep expectations clear.
-
Agree that those are required for a real instance, I was just trying to understand if we can provide a package where the app does not crash on initial startup until those are set up.
All in all since we at least need changes to the TLS addon to also provide the root domain certs, we are looking at least to a package release after the next Cloudron version only. Just to keep expectations clear.
@nebulon understood. If needed, I'm sure we could make it not crash, even if it's not really a functional XMPP server. And for me I'm at least happy I have a way to run XMPP on my server. I'm just trying to share the work with others.
When is the next version scheduled for release?
-
We are currently quite busy with Cloudron 9 development, lets see if we can get some of those things still in.
After reading up a bit on prosody docs, I still dont fully understand what we should do with the default location and what we could serve up as http (currently it shows a 404 error page for example). I saw there is also BOSH which is intended for http, so maybe that can be placed there instead, but so far I don't have a good grip on prosody's use of domains, certs and ports.
-
We are currently quite busy with Cloudron 9 development, lets see if we can get some of those things still in.
After reading up a bit on prosody docs, I still dont fully understand what we should do with the default location and what we could serve up as http (currently it shows a 404 error page for example). I saw there is also BOSH which is intended for http, so maybe that can be placed there instead, but so far I don't have a good grip on prosody's use of domains, certs and ports.
@nebulon In general, XMPP isn't going to actually serve any web pages. I installed an extra module for health checking because Cloudron requires it - but there's no reason it needs to be exposed externally. BOSH may use HTTP, but it doesn't serve responses that can be interpreted as web pages.
Personally, I don't think it hurts that there's a 404 on an HTTP/S port for an application that doesn't serve HTTP. Putting up a big page like "You found an XMPP server!" just invites trouble from the bad people on the internet.
-
Thanks for that input. I think what we really need to add is a way to run apps as headless services. We already have a few of those like game servers and synapse (matrix) which have to serve up a placeholder page. That is not ideal indeed. After Cloudron 9 we will also look into other things like maybe databases, they would have very similar requirements then.
Sorry that this is not yet implemented but we will get there in the future, so it is already great to collect requirements for such service type apps like prosody. This helps to avoid developing the wrong things on the platform side
-
Doesn't Matrix also require something on the root domain? Isn't that where the .well-known endpoint lives?
Something similar can be had?
@djxx, was the architecture chosen to use subdomains vs different ports?
If we're clever with the reverse proxy or with the main app server that listens to connections, it can ferry connections for other services by the request type or layer it needs to access, avoiding the need for extra ports or subdomains.
As for the HTTP page, why not make it multi-useful. For example, by default it's just a Cloudron OIDC login button and then per app it's configured what it does.. for backend services it could be a simple CLI stats page in HTML, or even a web terminal. That way it's flexible and useful to the admin, users, packaging and platform maintainers.
What else would be useful there?
-
Doesn't Matrix also require something on the root domain? Isn't that where the .well-known endpoint lives?
Something similar can be had?
@djxx, was the architecture chosen to use subdomains vs different ports?
If we're clever with the reverse proxy or with the main app server that listens to connections, it can ferry connections for other services by the request type or layer it needs to access, avoiding the need for extra ports or subdomains.
As for the HTTP page, why not make it multi-useful. For example, by default it's just a Cloudron OIDC login button and then per app it's configured what it does.. for backend services it could be a simple CLI stats page in HTML, or even a web terminal. That way it's flexible and useful to the admin, users, packaging and platform maintainers.
What else would be useful there?
@robi I'm not an expert, but I did do some research on this to respond to Nebulon above. The main thing to note here is that the XMPP protocol expects different domains for different functions. So yes - it's an architectural choice that is mostly beyond our control. Each subdomain represents a different module, and the modules talk to each other (and other servers). They need a way to identify themselves, and they use the domain.
I did some reading and it seems it is theoretically possible to make a server that utilizes different ports to differentiate the modules under a single sub-domain, but this would require tweaking and re-compiling an XMPP server.
Even if this was done, and all XMPP protocols followed, there's always the chance that some not-fully-compliant client that works with every well known XMPP server would not work with ours because we've chosen to deviate so far from the norm, while still technically following the standards. Making these upstream changes would require someone much more familiar with XMPP servers than me.
While I look forward to Cloudron allowing us to package more complex services, I think XMPP is in a pretty good place for the above-average admin; it currently takes less than 5 minutes to set it up. I will probably end up making a cron job to sync the certs into my app's storage volume when I deploy this to production. If I think it's useful to others, and Cloudron 9 isn't out yet, I'll share it here.
-
@robi I'm not an expert, but I did do some research on this to respond to Nebulon above. The main thing to note here is that the XMPP protocol expects different domains for different functions. So yes - it's an architectural choice that is mostly beyond our control. Each subdomain represents a different module, and the modules talk to each other (and other servers). They need a way to identify themselves, and they use the domain.
I did some reading and it seems it is theoretically possible to make a server that utilizes different ports to differentiate the modules under a single sub-domain, but this would require tweaking and re-compiling an XMPP server.
Even if this was done, and all XMPP protocols followed, there's always the chance that some not-fully-compliant client that works with every well known XMPP server would not work with ours because we've chosen to deviate so far from the norm, while still technically following the standards. Making these upstream changes would require someone much more familiar with XMPP servers than me.
While I look forward to Cloudron allowing us to package more complex services, I think XMPP is in a pretty good place for the above-average admin; it currently takes less than 5 minutes to set it up. I will probably end up making a cron job to sync the certs into my app's storage volume when I deploy this to production. If I think it's useful to others, and Cloudron 9 isn't out yet, I'll share it here.
@djxx Appreciate the response.
We already have some apps that require a secondary domain on the front end, but that isn't necessarily the case for the backend. We just need Nginx to route things as expected by the clients, etc.
Glad you're here, packaging and interested in this stuff working.
