Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?

[d19dotca]

@girish I think (my two cents anyways) part of the issue may be the lack of tutorials / concrete guides with straight-forward examples of how to package apps which is leading to an ever-growing list of app requests faster than we can package them. I know I've had a few apps I wanted to tackle packaging to learn and train myself, but the lack of resources (and lack of time on my part) is a bit of a hindrance.

So I'd like to propose one aspect of a possible solution... eventually divert some resources into coming up with a neat playground / well documented area for how to package apps and even a few example walkthroughs (i.e. how were popular apps like Invoice Ninja and WordPress packaged, Umami, etc). I know we can get through the Git and see it ourselves but I think a bit of hand-holding such as a written or video walkthrough on how it was packaged (i.e. what was the starter template, and how each step progressed) would be awesome. I know it'd help me at least and presumably others. I know some of the steps are documented and there's some template apps to work off of, but I think it could still be made easier by having some detailed walkthrough guides with popular examples.:-)

That way more people could contribute easier to the app packaging process to fit Docker-ized apps inside of Cloudron's ecosystem.

[luckow]

Oh nerds. A lot of technical thoughts

Please allow me: 2 steps back.
What is the intent of the original question? Is it a general frustration with the lack of time between an app request and a Cloudron app release (like a child waiting for Christmas)? Is the intent to have more things to play with or to compete with other apps in the same category? Is there a real need for a missing "business related" app?

Are we really missing some applications? And if so, how could we get a clear overview of the missing categories? Do we really need a third or fourth web analytics or RSS reader app in the App Store? And if so, why? IMHO, the answer should not be: because we can.

How can we find out if an app from the app wish list is worth investing time to package it as a Cloudron app with all the benefits we need as a reliable app for our daily work?

To try out apps, I have a dedicated VPS for Docker containers. I usually follow the installation instructions in the Github repository and can usually try the app after a short time. My experience is: after a short time I run into some issues where I decide that reading the announcement and playing with the app contradicts my own expectations. But sometimes I like what I get. One of my recent discoveries was Gitpod. After spending more time with Gitpod, I realized that it's not a perfect fit for Cloudron because it's very dynamic (and resource hungry) when you share the new tool with your teammates. The same goes for BigBlueButton, which is on the app wish list, but it's not worth investing time in packaging.

For me, Cloudron massively reduces my personal time spent on business critical applications. Kind of a "fire and forget." To be fair, most of the time I spend on new applications is configuring the tool, documenting it, and explaining it to my teammates. Once that's done, I forget about it until the next major release comes out, and I have to invest time to get an idea of the new features. But all that crap about updates, backups, reliability .... That's why I decided to subscribe (to pay people for their work).

Have you ever looked into a random docker.hub image? Have you ever looked into updating a random image? In my opinion, sometimes things go wrong, and sometimes they don't. So I know that mission-critical apps take time to understand, plan, and maintain. With that in mind, I've decided not to put some "cool new kid" on the app wish list. I invest time to get an idea of whether the app is worth investing time to package as a Cloudron app.

Maybe we should create a new forum category "cool new kids" where we can showcase new apps we've heard about. From there, we can invest some time (as a community) to find out if the app is worth investing time to package as a business critical app (aka Cloudron app)

[micmc]

@d19dotca said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

So I'd like to propose one aspect of a possible solution... eventually divert some resources into coming up with a neat playground / well documented area for how to package apps and even a few example walkthroughs

I know it'd help me at least and presumably others. I know some of the steps are documented and there's some template apps to work off of, but I think it could still be made easier by having some detailed walkthrough guides with popular examples.:-)

That way more people could contribute easier to the app packaging process to fit Docker-ized apps inside of Cloudron's ecosystem.

I agree, from there I'd certainly contribute to package apps myself too. It's the same thing for me, the lack of time to dig deeper to figure this out with little information. I've been administrating and managing web server for 2 decades, and started to master docker and cloud technologies on top of that about 5 years back, and I'm always on the fence of getting onto try to package apps for Cloudron, however for the same reasons mentioned, when I try to get onto it then too much question pop and I've to postpone the try because of lack of time to play to figure it all.

[micmc]

@luckow said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

Oh nerds. A lot of technical thoughts

Maybe we should create a new forum category "cool new kids" where we can showcase new apps we've heard about. From there, we can invest some time (as a community) to find out if the app is worth investing time to package as a business critical app (aka Cloudron app)

That's a good idea and maybe that would help put aside not only the "cool new kid" to take a look at, but also that 10th RSS reader which the new Cloudron user who just comes in, wish to have on his Cloudron because it's the one he knows and prefer. Indeed, we don't need 5 packs of each good apps out there so then we should concentrate on getting at least one good app, preferably the best one available, for each of the category we'd consider an asset to put on Cloudron that would enhance the offer.

[robi]

@girish said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

I think this is what sysbox solves, but I don't have much experience with that.

Yes, it takes 10m or so to try it out.

We can circle back with @Rodny-Molina if needed.

[eddowding]

One solution might be to have a marketplace / bounty space.

1] Packages have a crowdfunded bounty behind them. Want it built? Chip in £50.

But this has a disincentive to speed, since why build for £5 when you can wait a few months for frustration to mount, and build for £500?

2] Have a month "pay per app" fee for apps which is Pay What You Want with a minimum of £1/m until a reasonable fee has been reached.

Some blend of that (see also opencollective.com) seems like it might make a viable solution.

[marcusquinn]

@girish It's a lot about speed of research before development.

As an example, we just got OpenDroneMap running the same day on a separate VPS with the Docker Compose scripts.

Doing the same thing with Cloudron would be impossible due to the multiple Docker Containers it relies upon.

With some cooperation we could package it for Cloudron - but we just don't have the week of head-banging time to add to developments, even if we can convince you to make allowances for all of this app's needs, we need to have something working and move on to the next thing.

With what I'm suggesting, we can have a VM (LXD or equivalent) container app up and running the same day, just as with any separate VPS, and we can share that.

I can't see a downside, and it's just using features that Ubuntu already offers.

We'll also help with any port-conflicts feedback or anything else that might cause issues in using the same VPS.

You have allies here, we just don't have all we need available from Cloudron, and any time we lose in battling things that won't work quickly, is time away from the additional app packaging we could be helping with for the overall cause.

[marcusquinn]

@d19dotca This only helps for apps that will work with Cloudron as-is. Our main issue is apps that require changes to Cloudron itself, they are literally impossible to package with current restrictions. I understand the restrictions, and their reasons, but they are real blockers, so we know before we have started that we will fail, so all effort then goes into separate VPSs, which is of no benefit to this ecosystem, and those VPSs can't benefit from any of this ecosystem either.

[marcusquinn]

@luckow No - the point is the apps we are needing cannot be packaged for Cloudron at-all as it is. The main ones are multi-container apps. There's many apps where discussion has acknowledged they need add-on apps within apps, and then they cannot progress until that is a feature.

In the meantime, we could have been packaging way more apps, just without all the integration features, but in a way that they would at least be working and demonstrate-able for then showing exactly what they can do and what they would need from extending CLoudron's full-integration app packaging framework to allow for.

[marcusquinn]

@micmc Nothing to do with having every app under the sun - this is about having apps that are otherwise impossible to package for Cloudron - so likely there is not a single instance of what they offer. We'll package a lot more Cloudron apps - but we don't have time to negotiate all the things Cloudron needs to be allowed or extended to do, we just need to have an environment that lets us get on and focus on the packaging. This does that, that's why I'm suggesting it.

[marcusquinn]

@eddowding That's an option for motivating packaging, but it ads admin overhead IMHO. The issue we have is that we literally can't package so many apps due to Cloudron restrictions that having VM Container Apps would solve, and then speed up demonstration of the needs to extend Cloudron's native app packaging framework.

[timconsidine]

@robi total newbie to sysbox here.
Is it a case of run sysbox on Ubuntu, then run Cloudron in one sysbox container and <a.n.other.docker.app> in another sysbox container ?

[timconsidine]

@micmc I agree, as I've commented somewhere above.

The packaging documentation is good, but doesn't particularly help new packagers on their journey.

We need more examples, walkthroughs, even boilerplates.

I understand that's quite a burden for busy staff.
I'm going to knock up a wiki, but welcome contributions.
Especially with the right answers !

To kick it off :
https://forum.cloudron.io/topic/7087/packaging-own-apps-what-guidance-do-you-want

[timconsidine]

@robi I just noticed this while exploring

https://www.docker.com/blog/docker-advances-container-isolation-and-workloads-with-acquisition-of-nestybox/

Any thoughts on impact / future ?

[robi]

@timconsidine said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

@robi total newbie to sysbox here.
Is it a case of run sysbox on Ubuntu, then run Cloudron in one sysbox container and <a.n.other.docker.app> in another sysbox container ?

No, much simpler and more elegant. (that is also possible though)

Run Cloudron, install sysbox, configure docker to use sysbox instead of runc (default runtime), done.

Now all new containers benefit from the advancements of sysbox along side any others already running.

Of course this can be added to Cloudron as a selective option upon app install from the App store as well as support running non-packaged apps from Git* hubs or Docker container hubs.

This keeps the server/VM looking like bare metal as much as possible. From there you can play other VM encapsulation schemes as you normally would.

Check the previous threads on Sysbox where docker-in-docker and Cloudron-in-Cloudron are discussed.

[robi]

@timconsidine said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

@robi I just noticed this while exploring

https://www.docker.com/blog/docker-advances-container-isolation-and-workloads-with-acquisition-of-nestybox/

Any thoughts on impact / future ?

OH MY GOODNESS!

This is great news, thanks for finding it!
Congrats to @Rodny-Molina and Ceasar.

This further solidifies the sysbox ideas, implementation and product as a key part of Dockers mission.

Good on the Docker team to see this and bring them in-house. Sysbox is here to stay.

Long live Sysbox.

[girish]

@timconsidine said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

https://www.docker.com/blog/docker-advances-container-isolation-and-workloads-with-acquisition-of-nestybox/

Oh, didn't know! Wonder if we will see a 1.0 soon then.

[marcusquinn]

OK, so it seems we have both interest in this - and an understanding that it's a capability that significantly lowers the barrier to entry for having apps created — albeit without all the Cloudron integration features — although I think Location could still work, Email is usually easy to configure manually, and it gets apps to being at least self-hosting testable a lot faster for proof-of-concept research & development.

@girish I honestly think this Nestybox as an App feature will save you a LOT of time from app packaging, because we'll all be able to get a lot more done faster this way, and submit them for commissioning as full citizens when time, appetite and functionality of the Docker alone way of packaging is more desirable.

[micmc]

@robi said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

@timconsidine said in Why Cloudron's Docker only? How about VM containers with generic Docker Compose scripts?s?:

@robi I just noticed this while exploring

https://www.docker.com/blog/docker-advances-container-isolation-and-workloads-with-acquisition-of-nestybox/

Any thoughts on impact / future ?

This further solidifies the sysbox ideas, implementation and product as a key part of Dockers mission.

I was going to point it out and let you know, and boom. Yes, and this system sounds much more as a solution and clarifies what was intended and proposed by Marcus @marcusquinn to address the actual concern to being able to run certain apps that potentially could 'never' run under Cloudron because of its own infrastructure.

The sysbox reminds me of the Qubes OS which is also recommended (E. Snowden) as the most secure desktop OS today because it runs every app in its own container.

[marcusquinn]

@micmc Nice! I've not seen that one before, might fire up an instance to explore. Also reminds me of Firefox Containers with the multi-coloured window frames.