I believe @Lonk 's VPN project may be helpful here as VPNs track their packets much more so than general network interfaces which could provide for greater visibility for any app by simply adding it to the VPN profile & data path.

Until @girish gets a more general solution in for all of Cloudron and perhaps containers in general.

My wording isn't quite correct, it's not full VMs. See below.

https://blog.nestybox.com/2019/09/13/system-containers.html

A Nestybox system container is an enhanced Docker container, designed to package not just applications but also low-level system software.

What type of system software are we talking about? Currently Systemd and Docker, but in the near future software such as Kubernetes, graphical display servers, and others.

The following figure illustrates the difference.

But can’t you do this on a regular Docker container? No you can’t. Not properly.

For example, in order to run Docker inside a regular container (i.e., Docker-in-Docker) you need to run the container in “privileged” mode. This significantly weakens isolation between the container and the underlying host, posing a strong security risk (especially if you don’t trust the workloads running inside the container).

But in some cases even privileged mode is not sufficient. For example, some system level programs read resource consumption information from the kernel (e.g., via the Linux /proc directory). In order for the program to work properly inside a container, such information must be provided relative to the resources assigned to the container itself, not the resources of the underlying host. A regular container does not do this, even when running in privileged mode.

Nestybox system containers are designed to solve these problems.

We can summarize the key properties of a Nestybox system container as:

Runs low-level system workloads (as well as applications).

Provides strong isolation from the underlying host.

Presents a more complete abstraction of a virtual host to its workloads.

Typically runs multiple applications within it (rather than just one app).

One way to look at it is that a regular container packages applications. In contrast, a Nestybox system container packages virtual host environments capable of running applications as well as system-level workloads.
See it work!

Use Cases

But why would you want to run such system-level software inside a container in the first place? I.e., Why do we need system containers?

There are several use cases.

For example, by virtue of running Docker inside the container (securely), the system container can be used for:

CI/CD pipelines (where the need for a container to run another container arises).

Docker sandboxing (e.g., to run multiple Docker instances with total isolation between them).

Our blog site contains articles with practical examples of such use cases.

In the near future, as we add support for more system-level workloads inside the system container, more use cases will open up.

In general, if you have a need for a virtual host that runs many of the same workloads that you could run on a VM, yet is faster and more efficient, then a Nestybox system container is a good fit.

Key Features and Benefits

Deployment with Docker (and Kubernetes)

This allows you to leverage the power of these amazing tools to build, deploy, and manage system containers. No need to learn new tools.

Fast & Efficient

Just like regular application containers.

Strong Container Isolation

Nestybox system containers always use the Linux user namespace.

This means the root user in the system container has full capabilities inside the system container, but none outside of it.

In addition, Nestybox system containers use exclusive Linux user namespace user-ID and group-ID mappings for each system container.

If a process inside the container escapes the container sandbox, it will find itself without privileges to access resources of the host or of other containers.
Image Flexibility

A Nestybox system container image can be created with Docker, just like any Docker container.

However, it typically is configured with an environment resembling a virtual host (e.g., process manager, multiple apps, docker, app containers, graphical display server, etc), although you can also configure it with a single system-level application (e.g., Docker) if you wish. It’s up to you to choose what’s in the image and the entry-point.
Portability

You can deploy Nestybox system containers on any Linux machine, whether it’s bare-metal, a local VM, or a cloud VM, in a data-center, your laptop, an edge device, or even an IoT device.

And as with any Docker container you have the flexibility to move the system container around as you wish. Just upload it to your repo and deploy it on the target machine with Docker.
Partially virtualized procfs

In Nestybox system containers, portions of the Linux procfs (/proc) are virtualized. The goal is to make the system container more closely resemble a real host or VM. For example, the /proc/uptime file returns the container’s uptime, not the underlying host’s uptime.

How does it work?

Nestybox system containers are made possible by Sysbox, our system container runtime.

Sysbox is software that installs on the Linux host machine, integrates with Docker (and soon Kubernetes), and works under the covers.

Users interact with Docker to create the system container image and deploy it, just as with application containers. The difference is that this image can now include system-level software such as Docker itself (for Docker-in-Docker), etc.

The following figure illustrates this.

Running the system container is simple, it only requires passing the --runtime=sysbox-runc flag to Docker:

$ docker run --runtime=sysbox-runc -it my-syscont-image

Under the covers, Sysbox takes care of setting up the system container abstraction so that it can properly run system level workloads.

It’s easy. And you avoid the need for unsecure privileged containers or complex container configurations.

Is it a VM?

No, it’s not. It’s an enhanced container. As with all containers, it uses OS-level virtualization and shares the Linux kernel with the rest of the system. In contrast, VMs use hardware-level virtualization (i.e., emulate hardware in software) and have a dedicated OS per VM.

The following figure illustrates the differences.

This gives system containers and VMs different properties. In particular system containers are faster, more efficient, and more portable (see above) but offer a lesser degree of isolation from the underlying host.

From a workload perspective however, Nestybox is working to make our system containers support as many workloads as VMs can run such that they can present a viable alternative to VMs in some scenarios.

@mehdi said in Ability to modify server name used for SFTP access to avoid use of Cloudron's my.<domain>.<tld>:

the prompt that everybody confirms without even reading

haha, so true. Yeah I got a pop-up that was like "This is a new key" or something and just accepted it. Basically the same kind of message that happens with SSH. I just have certificate-based SSH though where it needs my key to even connect (i.e. you couldn't connect to my server over SSH without it), so I was surprised I didn't need that on my SFTP connection, thought I'd maybe need something similar. But I guess this makes sense then the more I think about it. Just caught me off guard. haha.

@ianhyzy I registered a new domain just for this purpose. It's not ideal but I don't have to use a relay so it works out in the end. I'm hosting with DigitalOcean so the PTR record is set by changing the server (droplet) name to match the mail server domain. Last I checked my headers, all were good and pointing to the new domain setup.

Edit: BTW, I've used Amazon SES for my newsletters in the past and they're pretty cheap. IIRC, I sent like 9k emails for under $1 USD.

@atrilahiji F that, I just use Dark Reader extension on everything, does a better job in 99% of cases anyway.

@marcusquinn I think DNS automation (provider specific) atleast needs to come from elasticemail itself ideally. It's not possible to keep track of all the DNS records each provider wants to set and if it's even available with API. It's basically impossible to test on our side atleast.

Are you telling me that apt install screen is going to break updates? 😆

@girish hinted, that this needs more vetting here. So if that app would have been auto-updated, then a backup would have been made, regardless of automatic backups or not.

That in its own may or may not be what one would expect.

Either way to be consistent at least, I will change that to have the skip backup checkbox set by default only if automatic updates and backups are disabled.

Edit: pushed the new fix

For those checking this later, the apps that have automatic backups are disabled are listed in the Backups view (under Location).

5a24bfd9-5e53-42f7-b62a-e509e2ab4671-image.png

@atrilahiji Not sure, I just liked the look of it and thought could be useful, so this section seemed as good as any. Could be in Discuss as well I guess.

@nebulon would it be too much to make sure, either just from a visual stand point to encourage people to save that encryption password offline/somewhere ?

This is implemented in Cloudron 6 ! https://forum.cloudron.io/topic/3205/what-s-coming-in-6-0-take-2/99

@marcusquinn see my post about n8n at https://forum.cloudron.io/topic/1939/n8n-io-zappier-ifttt-integromat-alternative/13?_=1605126531115 - short version is that it's been packaged, it's just not primetime-ready

@robi said in Add new App notifications:

@lonk no, monthly doesn't make sense apart from an email, like the new Cloudron release emails.

Weekly maybe, but ideally the day it is released.

From there those interested in those can subscribe to the stable Apps or unstable Apps notifications.

I like that. Opting into app updates without having them installed. Often times user's don't want to be the early adopters (I do, but I'm me 😂) so they wouldn't install anything initially, and getting release notifications even though the app isn't installed might eventually prompt them to. It's not even too difficult a change since it already exists for installed apps.

@fbartels said in MySQL tuning with my.cnf settings optimisation:

@lonk said in MySQL tuning with my.cnf settings optimisation:

when do we run / apply it?

Mysql tuning is best applied after a few days of database usage. Applications usually have different load patterns.

Gotcha, thanks for the tip! ☺️

@nebulon My bad, I created a discussion in the discussion because I felt it was loosely related to password management.

@robi Installing apps from the store en masse can't be common. I think rn it's flow is best for an average user, but it could be a little more mobile app store like with an "installing" progress bar while you discover other apps, and then it pops up with "installed, click to open" or something via Notifications. I could see it, but again - installing apps isn't common enough I don't think to worry about this since installing is a one-time task.

But I'm glad you brought it up because it would add a bit of polish to have a more "app store" feel to it. Still allowing app discovery while an app's installing. I think Cloudron should be able to do that one day even if it doesn't feel like a priority rn.

@nebulon said in Extra fields in LDAP:

The phonenumber for Mattermost is one such use-case, but that can be also solved without custom fields.

Assuming there might be a need for this in the future - what was the solution you thought of?

Also, even if app's don't use the fields, just having the Cloudron User directory more fleshed out might be a good thing for reference sake.

thanks @fbartels!

Hey guys, sorry for the delay

I've updated the fork and the providers function, if I get permission to create a new branch or you open one I can open an MR with the new branch and let you review