I'm reworking this now that 6 is out and proxyAuth is an option. Currently got the latest version building and running on the latest node. Waiting for some details on how to do proxyAuth configuration to not break webhooks (rather important for this one!) and then I'll need to get to fixing up the websockets, but overall is looking okay. Switching from file-based to env-based configuration only because it seems better supported by the docs as well.
I had a breakthrough with the dumb websockets issue. Hooray! This one should be ready shortly. I know it's been highly requested and hotly anticipated (for me as well) so it's my top packaging to finish. At this point, after a few other tweaks, I'm fairly sure I've got this working. I'll do some final testing tonight, but I expect that I should have an initial build together by this weekend.
This is something I've been batting around implementation ideas on for a little while now. There's a ton of variability provider-to-provider to account for on automating some of it, so I was leaning toward more capable cluster managers that are already available off the shelf. Easily the most capable is Kubernetes, but it comes with a lot of added complexity. It's distinctly possible based on the way Cloudron works to entertain some of this stuff, and I've been sketching out a number of different ideas. Nothing is formally roadmapped afaik right now.
That said, it would be helpful in thinking about options what you see as the changes in your experience that these sorts of ideas would enable. Adding an additional node, which seems to be what 3 of your 5 ideas are focused on (load balancing, hot stand-by, and auto-scaling), may or may not be the best approach to minimize downtime depending on how the "normal" use would pan out. It's already not all that difficult to keep an alternate machine restored from backups as a standby, but given the way the system handles app-level failures, it's hard to say in what cases that would be useful; there's added difficulty in reversing that failover and keeping it real-time.
Ultimately, what probably does make the most sense to close the gap on those goals while not messing too much with the underlying architecture and existing packaging is some sort of coordinated cluster manager that keeps the single-container approach but allows the system to reallocate those app containers across k different servers. Something short of Kubernetes could achieve this pretty easily, but will need a lot of work to pull off. For these reasons, I've started looking more at Hashicorp's Nomad as a potential solution to the cluster management side of things, but I'm still in the very early stages of what a Cloudron implementation would look like. At its full potential, this could enable things like multi-region and even multi-provider deployments. Ideally, the details of managing this would be hidden away behind the Cloudron interface, but I've not even yet begun to start spiking out an actual implementation.
I'd love some more thoughts and feedback on the approach generally though!
So I have an initial packaging for this done at https://git.cloudron.io/jimcavoli/n8n-app
HOWEVER, I'm honestly not sure I'd be comfortable putting this software into production as-is. It's pretty weak on some security features, but maybe its not the end of the world for people and I'm just super paranoid. There seems to be an undocumented way to do JWT auth, which I'm not going to even try to get into that, since I'm only aware of it from looking at the config file handling typescript code. The only supported authentication right now is basic auth as documented - with a single username/password set by ENV or config file (I went with the latter since it's easier for end-users to edit in /app/data/.n8n/config by terminal).
Either way, everything basically works, and by launching via supervisord with the working directory set to /app/data/output we're able to contain its binary file writing to system (probably a bad thing to do in prod as well, but it works).
Custom nodes can be added to /app/data/custom, the config file is JSON at /app/data/.n8n/config, and the package writes cloudron-provided postgresql configuration to that file at launch.
Some stuff only works in Chrome. That's just an issue with the app. Something about they way they're handling web sockets means other browsers see "Connection lost" in the top right, which is where the Activation toggle should be. Most other things actually work fine, but that's necessary for executing workflows in-browser manually and activating them.
I packaged this one because it seems to have some popularity in asks, but it definitely deserves some due diligence before heading to production for anyone or the app store. It's also pre-1.0 so not sure how much is changing how fast, and the setup as is was a pretty delicate thing to get done. Maybe my concerns are overplayed, maybe not, but either way, there's something we can start to kick the tires on for this app now.
Not only is this an awesome find and great-looking app (wish I knew of this in May), but something I have a distinct use for. As such, I've begun an initial packaging effort.
I've extracted the following from one of my projects: https://gist.github.com/jimcavoli/b390565eb98f62faae821c83c8e87100
It has some added substitution for using secrets for all configurable values (I hard-coded a couple of them in the actual project), but is otherwise exactly what I use. It has a built-in cache registry that tries to minimize the build time, some simplistic retry logic, and as written is triggered when a new release is published on the repo. I've had a pretty good experience using it, so figured it might be handy for others, in case you've got some fully-custom app you're deploying from a private registry. The only built-in assumption is that you use the Dockerfile.cloudron naming scheme, but you could edit that as well if you like.
I recently stumbled across the MIT-licensed SPFToolbox project and it got me thinking a lot more about more comprehensive mail server features and management tools that would be good to have in Cloudron. SPFToolbox has some nice functionality for running checks for all sorts of things, but its blacklist checker is what started to get my mind running about that feature and potentially others that would be awesome additions to the Cloudron email system to help administrators manage deliverability and so on.
This probably starts to creep into a bigger re-vamp of the email management area of the product, but IMO since mail servers objectively suck to maintain, the ease of use of the Cloudron mail system is a huge selling point. While it's probably not realistic to target everything that services like MXToolbox do, but there could be some more built-in and automated to make life even easier.
Some thoughts out of the gates:
I'm just sort of thinking out loud here, and would like to see what others think about those ideas or any other ideas about the mail server here.
Likewise - this seems tantalizingly close to ready - and it's high up my list of things I'd like to get to deploying. What's the remaining to-do list or best way(s) to contribute at this point?
Started to try to package this, but stopped almost immediately since it's going to be a lot of effort/figuring out due to the many backing services, several of which are not readily available as adding in Cloudron today. List is in https://github.com/mistio/mist-ce/blob/v4.3.8/docker-compose.yml but includes things like elasticsearch, memcached, and rabbitmq. Obviously some could be put in-container with the app, but a couple are a bit too heavy-duty imo to do that to. Sticking a pin in it for the time being.
Love the conversation this has kicked off, and I think there's a few good things to consider here:
A lot of us are also running fairly beefy servers for large production setups and it seems disingenuous to rule out completely more resource-intensive operations from the core packaging just because some machines at the lower end couldn't support them. Making them optionally available, even subject to certain resource constraints and so on, is exactly what I'm hoping we can get to - a lot more control over the mail system so that administrators can choose how much in terms of resources to invest in that system. A combination of more inspectable, optional services available on the core offering as well as allowing external smtp-based components to slot into the workflow sounds like the best of all worlds.
That error message would seem to indicate that it failed on DNS lookup trying to pull the CAA to verify it - unusual for DNS to fail that way, but stranger things have happened.
I could host via Webex - easy streaming/recording and simple to operate with a decent feature set. I won't likely be able to be the one doing any of the live work initially, but I could at least facilitate the meeting for everyone.
From English, this is where those keys show up:
appstore.accountDialog.intendedUse I'm really not sure about, but in English it's just "Intended Use" likely referring to the subscription tier of the Cloudron. Will need someone else to confirm on that one.
backups.configureBackupStorage.s3LikeNote only shows when you're configuring the backup location, choose S3 or an S3-alike backend and rsync as the format - it's the "Please remove any object expiration lifecycle rules since it will corrupt rsync backups." message here:
emails.eventlog.type.spamFilterTrained looks like this (the part under "Details"):
Note: It took me a minute to put this together while @nebulon was responding and I got pulled onto something else for a minute, but I think the detailed writeup is worth having for posterity, so I'll post it anyway.
So what's going on here is that the app in question isn't reading the "right" headers to find the remote address. Basically, the inbound requests come in and hit the box-level nginx reverse proxy, which forwards the request on with the original inbound IP in the X-Forwarded-For header. Since from the sound of it, you're just routing straight to your app in the container, you'll want to either reconfigure your logging library to use the forwarded IP header as the client IP or drop nginx or similar as a reverse proxy in front of your app and configure it to rearrange the incoming headers as your app needs.
Sounds like you can just adjust a configuration so that this (your existing flow) works nicely:
Basically, here, the headers are adjusted in the "Step 1" processing as they reach the Cloudron so when they reach your app, the proxied headers have already gone into place. Again, this configuration should be fine with the configuration that @nebulon mentioned going in, since that should reconfigure your framework to read these adjusted headers correctly.
Failing that, or for apps with more complex setup or which aren't able to read those headers on their own by configuration, the solution is to further proxy those requests, by adding nginx or similar to take over the "Step 3" handoff and smooth out any specific details (like re-adjusting headers) for the app, and having it proxy those requests down to your app, all in-container, so that the logging and such in your app will all match up with expectations/reality. The whole point of the second reverse proxy when it's added it to make the world appear as it needs to for the app and/or its components inside the container.
This would typically be done via the scheduler add-on in the app packaging. It wasn't implemented initially since Cloudron does its own automatic backups of the site. Is there an additional advantage to having these backups as opposed to just using the built-in Cloudron ones? I believe you can still manually create one if needed for portability between installations
@girish Just to double-check, the proxyAuth changes also include the path-exclusion logic, right?
Seems like more of a build toolchain thing, which would sort of put it up to those of us packaging apps and/or perhaps the build service to pick up as a feature. Intriguing, but strikes me as high-effort for middling (at best) reward at first blush.
I totally understand the caveats, and this discussion is just what I was hoping to get to with the ask. I think that it being the user's responsibility to configure group mappings if they want to use that feature is a very reasonable approach that strikes the right balance of maintainability and functionality. It's worth noting to people that it's an advanced thing you should only be working on if you know what you're doing and that it's not something that Cloudron can or should support beyond basic information about how the LDAP server side works.
A lot of this varies app-to-app, and frankly, I think it's appropriate to leave the nuances of the choices a user might make about how those apps behave to said user. Some apps are super powerful here, like Metabase, which can basically automate all of the role/group assignments for you if configured right, ranging all the way back to EspoCRM, which is basically non-existent in its support for group mappings besides basic login filters, which is already better handled by Cloudron anyway, and plenty of others like osTicket that are somewhere in between.
I think good points have been raised about the security concerns and sync issues, and it's worth making it clear where the line is on what's provided vs. what's possible. Furthermore, as packages update into the future and more support for group mappings is added, it would be unreasonable imo to expect Cloudron packages to anticipate everyone's preferences/needs. Patterns of use will vary hugely based on how the systems are deployed and by whom. The points above all sounds like you've gotten a pretty clear idea over the years of this one coming up about how to equalize the flexibility and clarify the boundaries for expectations with such a feature, all of which sound exceedingly reasonable, whether groups are implicitly all LDAP or explicitly configured in or not (minor detail to my mind).
Okay, so this is back in my crosshairs to one extent or another, and that's good since I don't know of anyone else packaging on a regular basis who spends all day in Ruby like I tend to 
The main thing to point out here is that while I definitely can package this, Decidim itself is not an app, but a constellation of Rails engines (it's a Ruby on Rails framework thing - https://guides.rubyonrails.org/engines.html if you want to read), so it has to be packaged up inside of a host Rails app to actually be of any use. I think such a thing could be written and done to a fairly positive result, with reasonably easy maintenance, but that would be my main concern.
I expect this will likely not be something I'm ready to pick up for a while, so curious to gauge interest in this app specifically, as well as any other feedback about dealbreakers and/or maintenance concerns.
Probably can remove this one from the list - it's been carrying a "no maintenance" notice on it and has been abandoned code-wise since 19 December 2018, and that death-notice banner went into the readme 05 May 2020.
Or as a GIF:
On the plus side, decidim is still alive and well.