Alternative lightweight base image

[timconsidine]

@robi said:

The question is what the experience is like in the terminal.

in an app packaged with cloudron-minibase ?
as far as I can tell, it is the same as with cloudron/base.

[robi]

@timconsidine as in all the troubleshooting commands still exist?

[timconsidine]

@robi honestly, I haven't explicitly tested that dimension as the minibase was a test, with those questions to be answered through actually using it.

But at the end of the day, it is simply the official cloudron/base:5.1.0 without node, etc.

I doubt troubleshooting commands would be lost during that process.
And I haven't specifically and deliberately removed any.

[timconsidine]

In the spirit of sharing experience of using cloudron-minibase, I just updated my Community App rybbit.

• 1st update was for latest rybbit upstream using cloudron/base : built app image was 5.29Gb
• 2nd update replaced final multi-stage build with cloudron-minibase + Node : built app image is 2.51Gb.

50% saving in app image size !

[jdaviescoates]

@timconsidine said:

rybbit

https://ca.cloudron.io/app/rybbit could do with a bit more of a description imho (and a working screenshot)

[timconsidine]

@jdaviescoates will get onto it - thanks for the nudge - laziness during publication - sorry.

[timconsidine]

Summary web page for the lightweight alternative, explaining what's in and what's out :
https://communityapps.appx.uk/minibase

Also in preparation for new cloudron base image supporting ubuntu 26.04, moving name convention from cloudron-minibase:0.0.x to tcmbp132021/cloudron-minibase:ubuntu24.04 .

EDIT : minibase has a few extra components, so now 77Mb in size.

[timconsidine]

A neat "trick" for those who are cautious about not building with official base image :

FROM cloudron/base:5.1.0@sha.... AS builder
# Ubuntu 24.04 — full toolchain: Node, Python, nginx, supervisor, etc.
# Build everything here

FROM tcmbp132021/cloudron-minibase:ubuntu24.04@sha.....
# Ubuntu 24.04 — same glibc, same libssl, same everything
COPY --from=builder /what/you/need /where/you/need
CMD ["/app/code/start.sh"]

The built image is in all senses a proper cloudron build in the builder stage.
But the final runtime stage shrinks the image.
There is a level of caution and attention need to ensure that all artefacts are copied in from builder.
But you will soon know if anything is missing.

[girish]

I looked into this a bit . We can't remove so many of the packages. Mostly, it's for sharing between all the packages..

build-essential / gcc / g++, nginx-full, ImageMagick + GraphicsMagick, the whole -dev set (libssl-dev, libpq-dev, jpeg/png/xml, …), mysql/postgres/redis/sqlite/ldap clients, supervisor, git, vim, tini, yq, crudini, are still needed .

I have a new image now which is basically those packages and the hub size is around 300MB. Already something which was 2.3GB. But of course, have to see how this plays out in practice on a cloudron with many apps (because we reduce the sharing).

[timconsidine]

@girish 300Mb is still a world of difference, so I see that as very useful.

As you say, the practical impact may not be huge in practice because of base image layer sharing.

It’s all about build and push/pull times.
And horses for courses. I may have turned tuned aggressively but even your slimmed list has some stuff which may never be used in some apps.

For example a simple go site with data fetched from json, not even SQLite.

[timconsidine]

@girish my last response was typed while waiting in an IKEA queue !
Better quality would be :

Really appreciate your feedback and the effort put into the 300MB 'slim' version, especially when your workload is so heavy with 10.x.x.

I guess the core difference is you are constructing a "platform package" while I was aiming at a "runtime only" image, with the inescapable consequence that the packager adds what they need for that specific app, even if that is more work for them.

For a platform package, your comments make total sense, and of course I defer to your experience and objectives. And 300Mb is a clear win over 2.3Gb with still quite a lot of convenience 'nice to haves' included.

My argument for excluding much of what you mentioned is :

• runtime package doesn't need much of it
• why include mysql and postgres when neither are needed for a specific app
• keeping a lean attack surface ( the less in it, the less that can be abused)
• it is rare that compile stuff is needed at runtime, maybe include that in a multi-stage "builder" stage then omitted in a final "runtime" stage.

For example, a simple Go or Python application that doesn't perform image processing should not be forced to pull ImageMagick or gcc just to run.

My cloudron-minibase does push a lot of responsibility onto the packager.
That may not be popular, but they don't have to use it !
And I totally understand a bit bigger image to meet your goals about platform experience is valuable.

As a test, I updated cloudron-minibase to include :

• tini and supervisor
• libssl-dev and libpq-dev
• yq and crudini
• nginx-full

This gives an image size for cloudron-minibase:0.0.2 of 77Mb.
(FROM tcmbp132021/cloudron-minibase:ubuntu24.04@sha256:ae1e4ab4343f9f9dff05b988db94c8936f8390b8d1e88db80a1ae7afde563aea).

This keeps the base significantly smaller than the 300MB version while providing the tools that 90% of apps actually need at runtime. For the remaining 10% (the ones needing heavy compilation or ImageMagick), the package developer can simply add those specific layers to their own package, build stage or runtime stage. Or simply use your future "cloudron/base-slim" if you decide to release one.

So actually I am totally with (a) following your guidance as to what is best, and (b) using a 300Mb image for simpler packages and (c) continuing to use the full cloudron/base of 2.3Gb where it is justified.

I didn't intend cloudron-minibase to be a disruption or a challenge.
The inital 50Mb was insane, unexpected, and I would still have been happy if it was 300Mb. So no problem at all using a 300Mb base if one becomes available.

[girish]

@timconsidine said:

So actually I am totally with (a) following your guidance as to what is best, and (b) using a 300Mb image for simpler packages and (c) continuing to use the full cloudron/base of 2.3Gb where it is justified.

So the plan is use a new base image which will be 300MB and see how it goes. The dev packages are needed for many of the nodejs and ruby programs unfortunately. But it's a start and we can experiment a bit from here.

Really appreciate your feedback and the effort put into the 300MB 'slim' version, especially when your workload is so heavy with 10.x.x.

This was part of the 10.1 work. It brings a new postgres, dovecot etc. All good stuff

[timconsidine]

@girish awesome !

[jadudm]

From a security standpoint, leaving dev packages out of the runtime container is a Very Good Thing.

So, a question (with a clear angle): do nodejs and ruby programs need dev tools to run, or do they need dev tools to build?

If the former, I'm surprised. If the latter, then a multistage build allows for a slimmer and more secure runtime container.

I could also be confused, which is a common state of being.

[timconsidine]

@jadudm my understanding is dev tools only for building, so multistage build is the way to go, generally and from attack surface perspective.
But I’m no expert.

[jadudm]

@timconsidine In my day job, we wouldn't be allowed to ship production containers containing dev tools without an exception from an ISSO. I'm also not an expert, but I know that I am not allowed to ship production containers that provide attackers with tools to build further attack tooling from within my app container. (Perhaps that's just a +1.)

[girish]

@jadudm I think @timconsidine is referring to the dev libraries, compilers, toolchains, shared objects etc as dev tools. I guess you are refering to the likes of programs that can say launch some network attack?

[jadudm]

In this instance, I am referring to (say) build-essential on Debian. If I have a C compiler in my runtime image, and someone compromises the host, they can build further tooling in the host because I gave them a hammer and nails.

We generally have to follow NIST guidelines, and therefore our containers:

1. Are not writeable (generally)
2. Contain only the packages necessary to execute the software in the container
3. Require allow listing for egress (meaning nothing in the container can talk to the outside world); this implies an egress proxy
4. Only allow specific connections in (ingress proxy)
5. ...

Cloudron does some of this already. (To be clear: the reason I only use the official packages, and stay up to date, is because I trust the core product. Kudos to y'all, always.)

Hence, the story I'm waving (which I think Tim is as well) is that there might be a large, common build image for Cloudron (containing all of the build tools), but the runtime image (which a multistage build would copy into) is light/tight and shouldn't contain (say) a C/C++ compiler. You generally can't avoid having your language runtime(s) (e.g. ruby), but that doesn't mean that the runtime image needs the capability to install new libraries: everything should already be there for the app to execute.

Maybe that helps explain my thinking/comment.