Add ability to run VM like containers in Cloudron via Sysbox

[Lonk]

@robi said in Why do we have to push an image to a registry?:

Can't wait for Sysbox integration so you can have a dev cloudron running on any prod Cloudron in a container, completely isolated.

What, you're kidding me, that would help so much with box code changes. I've been making all of my box code changes in ssh via nano and...well, that method leaves something to be desired with how many changes all the files went through.

It's this "Sysbox integration" planned? Still not sure exactly what Sysbox is, but if it allows me to contribute more easily to Cloudron box code, I'M INNNN!!!

[robi]

Yes, I posted about it.

[mehdi]

It is not planned, I believe, as the devs have not given their opinion in the matter.

Also, one could technically already create a Cloudron-in-cloudron app today. The limit is not the containers, as you can run docker-in-docker. The limit is that it would be an infrastructure nightmare, like which cloudron should expose which ports on the main public IP, and such. Sysbox would not help at all in this matter.

[mehdi]

I do not think Cloudron should move to Sysbox.

• It is not necessary for cloudron apps to run "full systems" instead of just an app inside the container. I do not see any usecase for this
• ~350 github stars vs ~7300 => it's less maintained, there's less community : let's stick to the regular runc everyone uses, it will be easier to solve problems when they arise

[Lonk]

@mehdi said in Add ability to run VMs in containers in Cloudron via Sysbox:

The limit is that it would be an infrastructure nightmare, like which cloudron should expose which ports on the main public IP, and such.

Well, running Cloudron inside of Cloudron as an app would just massively speed up development. That would be my only reason, so
wouldn't care at all about security. Just development speed.

[Lonk]

@robi What does sysbox allow for beyond @mehdi's suggestion of just adjusting Cloudron itself into having the ability to be inside it's own container - which for me would allow for box code contributions to use the same development flow as my app development flow - so I'd contribute more to the project, given it is open-source.

I don't think I'd need something like Sysbox if that's all I want it for, right?

[Lonk]

Yeah, I reread this and I see sysbox of just being needless overhead when Cloudron can just be adapted as an app (to easier develop on, it the developers ever alright that). Reminds me of the whole "you can only emulate the Apple OS on Apple Hardware" thing. But it would be cool if I could alter Cloudron a little bit to give it the ability to run inside another Cloudron. The dev benefits alone, so much more speedy for box changes. At least for external contributors. I don't have a build flow for box so if we're ever able to use our normal cloudron update for it, I'll be contributing a lot more to this project with feature forks.

[robi]

Who doesn't want strongly isolated containers without having to run actual VMs?

Nestybox empowers containers to act as virtual servers capable of running the same workloads as VMs (e.g., Systemd, Docker, Kubernetes, and even legacy apps).

​Currently this requires unsecure privileged containers plus complicated Docker images with tricky entrypoints and custom volume mounts.

No more.
Nestybox enables you to do this using:

• Simple Docker commands
• Simple Docker images
• Strongly Isolated Containers
• No Hardware Virtualization (VMs)

Use Cases:
Kubernetes-in-Docker

• Running Kubernetes clusters inside containers is very useful for development, testing, and CI/CD.
• It avoids the need for heavy and costly VMs or cloud-based clusters.
• There exist a few tools to run Kubernetes-in-Docker. However these use complex container images and very unsecure privileged containers.
• Nestybox fixes this, enabling you to deploy the cluster in containers using strong isolation and very simple container images that you fully control.

Lightweight VM

• Sysbox makes it easy to use containers as lightweight VMs. For example, a container image can include systemd, ssh, a Docker daemon, preloaded inner container images, etc. You have full root access inside the container, but no capabilities outside of it.
• You can pack 2x as many containers as VMs on the same machine and get the same performance. And you can provision them 10x faster than VMs.

Docker-in-Docker

• It's often useful to run Docker inside a container for development, testing, and CI/CD.
• Up to now, the only way to do this was to use very unsecure privileged containers or exposing the host's Docker socket into a container. Neither is ideal.
• Nestybox removes these limitations, enabling you to run Docker inside a container with total isolation from the host.
• You can even preload inner container images into the outer container using a Dockerfile or Docker commit.

Legacy Apps

• With Nestybox, legacy apps may be lift-and-shifted into containers, enabling them to operate within cloud-native frameworks without resorting to VMs. This voids the need for re-architecting such applications.

[Lonk]

@robi Okay, I'm interested - compare the current system to this proposed system with some pros and cons?

[robi]

@lonk That would be great, try it out and see what breaks and where there are gaps.

[Lonk]

@robi No no, I'm asking, what does this give us in a practical sense and how hard would it be to implement do you think?

[robi]

@lonk maybe read the thread again?

[Lonk]

@robi So the pro is you can run your own OS completely inside a Docker container?

[robi]

@lonk yes, that was mentioned.

[Lonk]

@robi So there’s more. But the developers seem against it. Can you tell why?

[robi]

@lonk against it? where does it say that?

[murgero]

@lonk said in Add ability to run VMs in containers in Cloudron via Sysbox:

No no, I'm asking, what does this give us in a practical sense and how hard would it be to implement do you think?

imho - What I can see down the road is the ability for companies to run some applications without the need to officially packaging the app. This can be useful for in-house apps that use parts of the filesystem that is normally read-only for example.

[robi]

@murgero Yes, that is what is meant by the Legacy Apps point above.

[Lonk]

@robi Does it accomplish this by running another layer on top of the already existing Docker layer then?

[murgero]

@lonk I believe sysbox is a different container engine?