Add ability to run VM like containers in Cloudron via Sysbox

[Lonkle]

@mehdi said in Add ability to run VMs in containers in Cloudron via Sysbox:

The limit is that it would be an infrastructure nightmare, like which cloudron should expose which ports on the main public IP, and such.

Well, running Cloudron inside of Cloudron as an app would just massively speed up development. That would be my only reason, so
wouldn't care at all about security. Just development speed.

[Lonkle]

@robi What does sysbox allow for beyond @mehdi's suggestion of just adjusting Cloudron itself into having the ability to be inside it's own container - which for me would allow for box code contributions to use the same development flow as my app development flow - so I'd contribute more to the project, given it is open-source.

I don't think I'd need something like Sysbox if that's all I want it for, right?

[Lonkle]

Yeah, I reread this and I see sysbox of just being needless overhead when Cloudron can just be adapted as an app (to easier develop on, it the developers ever alright that). Reminds me of the whole "you can only emulate the Apple OS on Apple Hardware" thing. But it would be cool if I could alter Cloudron a little bit to give it the ability to run inside another Cloudron. The dev benefits alone, so much more speedy for box changes. At least for external contributors. I don't have a build flow for box so if we're ever able to use our normal cloudron update for it, I'll be contributing a lot more to this project with feature forks.

[robi]

Who doesn't want strongly isolated containers without having to run actual VMs?

Nestybox empowers containers to act as virtual servers capable of running the same workloads as VMs (e.g., Systemd, Docker, Kubernetes, and even legacy apps).

​Currently this requires unsecure privileged containers plus complicated Docker images with tricky entrypoints and custom volume mounts.

No more.
Nestybox enables you to do this using:

• Simple Docker commands
• Simple Docker images
• Strongly Isolated Containers
• No Hardware Virtualization (VMs)

Use Cases:
Kubernetes-in-Docker

• Running Kubernetes clusters inside containers is very useful for development, testing, and CI/CD.
• It avoids the need for heavy and costly VMs or cloud-based clusters.
• There exist a few tools to run Kubernetes-in-Docker. However these use complex container images and very unsecure privileged containers.
• Nestybox fixes this, enabling you to deploy the cluster in containers using strong isolation and very simple container images that you fully control.

Lightweight VM

• Sysbox makes it easy to use containers as lightweight VMs. For example, a container image can include systemd, ssh, a Docker daemon, preloaded inner container images, etc. You have full root access inside the container, but no capabilities outside of it.
• You can pack 2x as many containers as VMs on the same machine and get the same performance. And you can provision them 10x faster than VMs.

Docker-in-Docker

• It's often useful to run Docker inside a container for development, testing, and CI/CD.
• Up to now, the only way to do this was to use very unsecure privileged containers or exposing the host's Docker socket into a container. Neither is ideal.
• Nestybox removes these limitations, enabling you to run Docker inside a container with total isolation from the host.
• You can even preload inner container images into the outer container using a Dockerfile or Docker commit.

Legacy Apps

• With Nestybox, legacy apps may be lift-and-shifted into containers, enabling them to operate within cloud-native frameworks without resorting to VMs. This voids the need for re-architecting such applications.

[Lonkle]

@robi Okay, I'm interested - compare the current system to this proposed system with some pros and cons?

[robi]

@lonk That would be great, try it out and see what breaks and where there are gaps.

[Lonkle]

@robi No no, I'm asking, what does this give us in a practical sense and how hard would it be to implement do you think?

[robi]

@lonk maybe read the thread again?

[Lonkle]

@robi So the pro is you can run your own OS completely inside a Docker container?

[robi]

@lonk yes, that was mentioned.

[Lonkle]

@robi So there’s more. But the developers seem against it. Can you tell why?

[robi]

@lonk against it? where does it say that?

[murgero]

@lonk said in Add ability to run VMs in containers in Cloudron via Sysbox:

No no, I'm asking, what does this give us in a practical sense and how hard would it be to implement do you think?

imho - What I can see down the road is the ability for companies to run some applications without the need to officially packaging the app. This can be useful for in-house apps that use parts of the filesystem that is normally read-only for example.

[robi]

@murgero Yes, that is what is meant by the Legacy Apps point above.

[Lonkle]

@robi Does it accomplish this by running another layer on top of the already existing Docker layer then?

[murgero]

@lonk I believe sysbox is a different container engine?

[Lonkle]

@murgero said in Add ability to run VMs in containers in Cloudron via Sysbox:

@lonk I believe sysbox is a different container engine?

Oh, now that I re-look at everything. You're right, I think it's too late for a restructure now.

[robi]

@murgero No.

It's simply a different container runtime.

Docker remains the same, we just tell it to use sysbox vs the default runcby adding --runtime sysbox-runc to the docker command line or default config.

That's it.

Simple.

[murgero]

@robi said in Add ability to run VMs in containers in Cloudron via Sysbox:

container runtime.

isn't that the same thing as engine? Or is docker the engine and containerd is the runtime?

[robi]

@murgero said in Add ability to run VMs in containers in Cloudron via Sysbox:

isn't that the same thing as engine? Or is docker the engine and containerd is the runtime?

No.
Docker Engine is a product name that uses containerd (the container daemon) which relies on runc (run container) which is a CLI tool for spawning and running containers according to the OCI specification.

All have a different abstraction level.

Therefore sysbox-runc is an alternate runc that is more secure and offers all of the above benefits.

Docker Engine and containerd don't change, and accept a parameter to specify which runtime (runc) to use.