Sysbox integration in progress..

[robi]

As a very happy birthday present, this came about synergistically, and now we have the worlds first Cloudron App running in a Sysbox container runtime from Nestybox.com

Gratitude to this community and the Nestybox team.
This couldn't happen w/o @marcusquinn & @Rodny-Molina

Marcus provided the Cloudron based development environment which made this super easy and convenient, and Rodny provided the integration muscle and Sysbox expertise.

Here is the first issue filed related to our work:
https://github.com/nestybox/sysbox/issues/151

Feel free to follow along or jump in.

That's all for now

[girish]

@robi Great progress! Looks like the readonly containers are causing some headaches. But it seems there is some fix for this already and will be merged soon.

[robi]

@girish the RO issue was expected, and an easy fix.

@Rodny-Molina may need further clarification from you on the DNS issue.

[marcusquinn]

Awesome stuff - this community rocks!

[mehdi]

I must admit, I am still quite skeptical about this...

@robi, I know you wrote at length about what sysbox could do, but I do not really understand what precisely are you trying to package as an app right now, that's possible under sysbox but not under the normal docker runtime?

[robi]

@mehdi I did not say I was packaging anything. It's an integration right now and making sure all the apps we have run as expected.

[atridad]

@mehdi https://forum.cloudron.io/topic/1373/gitlab-runner-for-ci/11?_=1607705685190

I can speak for my use-case. I would absolutely love to see GitLab Runner as an app on here and this would make it possible.

[Rodny Molina]

@robi thanks for your kind words and for your time answering all my Cloudron questions.

The fix for this issue is in code-review at the moment, should be merged soon.

[marcusquinn]

@atrilahiji for GitLab Runner. I'm curious if it would be possible to run full Windows Server VMs in it too, we have a bunch of use-cases for that.

[marcusquinn]

Answered my own question; no, Windows containers won't run on Linux: https://stackoverflow.com/questions/42158596/can-windows-containers-be-hosted-on-linux

[robi]

@marcusquinn
But they will run on Windows.
I have experience with this if needed in Windows and in Linux.

[marcusquinn]

@robi Yeah, that's being worked on this weekend. I think it saves on Windows licence costs too compared to muti-VMs.

[Rodny Molina]

@girish Question ...

Have you guys considered the option of removing RO requirement for specific applications? I'm talking about system apps such as docker, systemd, k8s, podman, ci/cd tools, legacy-apps, etc. All that (and more) can be potentially offered to Cloudron users. But as you know, this software needs RW access to diverse sections of the rootfs (such as /run) to create pipes/sockets/dirs, etc.

The system container running these special apps is fairly secure by virtue of running within dedicated user-namespaces. Also, it's self-contained, in the sense that when you do a docker-commit you are not only capturing the outer sys-container image, but also the inner docker images; that's to say that you can customize these system-apps to your liking, and reduce instantiation latency to the minimum (no i/o needed to fetch inner images).

Please let me know when have a chance.

Thanks.

[Rodny Molina]

@robi just helped me realize that /run is already bind-mounted as RW, i had missed that. There may be other paths for which RW access is expected though, but i guess that's something that can be evaluated on a per-app basis.

[girish]

@rodny-molina Sure, it's possible to remove the requirement as more use cases come up. Cloudron is currently targeting installing web apps (SaaS equivalents) and not targeting infrastructure apps/system app. I think CI/CD and Jupyter Hub style apps can find sysbox useful though. BTW, did I understand correctly that I can run sysbox and runc runtimes side by side? It does seem like that but wanted to confirm . And is a new release planned soon with the readonly fixes? Would be great if we can also download binaries instead of deb packages.

[mehdi]

@girish I am not 100% sure it's doable, but instead of running Cloudron apps in sysbox, I think it would make a lot of sense to run a sysbox container as an addon service for apps that need to run docker containers, and run them inside the sysbox addon container.

[girish]

@mehdi right, I don't want to move everything to sysbox. Just the ones that want it. But I want to know if it's possible to run them both side by side.

[mehdi]

@girish I understand that. My point is maybe we should consider putting it in a separate service container, instead of the app itself

[Rodny Molina]

@girish @mehdi, you can definitely run Sysbox side-by-side along other runtimes such as runc.

Sysbox will exclusively interact with its own containers. You just need to program your orchestrator to make use of Sysbox for those containers for which you want enhanced security or extra functionality.

Ping me if any question.

https://github.com/nestybox/sysbox#using-sysbox

---
Note that if you omit the --runtime option, Docker will use its default runc runtime to launch regular containers (rather than system containers). It's perfectly fine to run system containers launched with Docker + Sysbox alongside regular Docker containers; they won't conflict and can co-exist side-by-side.
---

[Rodny Molina]

@girish said in Sysbox integration in progress..:

And is a new release planned soon with the readonly fixes? Would be great if we can also download binaries instead of deb packages.

Forgot to answer this one. Yes, we are about to start working on the next release (ETA ~ 2 weeks). Not sure about the binaries though, will get back to you later on this.

[girish]

@rodny-molina Having a binary would really help because usually the deb packages have a tendency to restart existing services and also automatically start their own services.

Does the debian package support this scenario - https://github.com/nestybox/sysbox/blob/master/docs/user-guide/install.md#Installing-Sysbox-without-Docker-restart ?

[Rodny Molina]

@girish, right, this 'hitless' scenario is supported by the installer as long as the expected attributes (e.g. bip, address-pools) are already configured in the docker config file. If they are not present and digested by dockerd, then the installer will restart docker.

I understand that you may need more flexibility for Cloudron's specific setup. Can we talk to have these installation details fully understood? (rmolina@nestybox.com).

[girish]

@rodny-molina will do. Give me sometime to play around with sysbox before we have a talk, so maybe after your release. I want to give it a try in a couple of our apps to understand how it all fits.

[robi]

From a recent discussion on sharing data between apps, this will be interesting.

On the persistence of Inner container images..

[robi]

A community update from TYPO3:

https://gitlab.typo3.org/core-testing/testing-infrastructure/
This is a 'infrastructure as code' repository for a gitlab-runner setup using sysbox, maybe this helps anyone looking into similar things:

• bare metal setup with ansible - gitlab-runner with docker executor and sysbox'd test execution in DinD

• Hetzner cloud docker+machine - gitlab-runner with docker+machine autoscaling with sysbox on workers