OpenSlides - digital motion and assembly system

[jimcavoli]

Repo with initial packaging: https://git.cloudron.io/jimcavoli/openslides-app

I'd regard this as pretty ready to go at this point - all packaging choices were in line with the production configurations recommended by the project. I've tested pretty much everything except SAML manually. User management is independent for the time being as described previously. Also notable is that after some testing, I found 524288000 left enough headroom as a memoryLimit floor that the app was usable without constant restarts for light duty (not going to service a 1000 person assembly, probably, but fine for prepping an event and then deciding how to scale up from there). Other notes are in the README of the packaging repo. Also recommend building with buildkit turned on if you don't normally - the JS compilation takes a short eternity, but the Dockerfile takes pretty good advantage of buildkit multistage builds for parallelizing as well as limiting bloat in the final container.

[girish]

@jimcavoli awesome, thanks! will take a look next week.

Marked this as wip.

[jimcavoli]

@girish Just a bump on this - figure it got lost in the 6.0-releasing shuffle. No huge deal, but hope it can still make unstable this year

[girish]

@jimcavoli Not lost It's marked as https://forum.cloudron.io/tags/wip . But yes, now that we got Cloudron 6 out of the day (save the announcements), we will have time to push these out.

[girish]

The build currently fails

Step 6/32 : RUN mkdir -p /app/{src,build} &&     curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src &&     chown -R cloudron:cloudron /app/src
 ---> Running in cb29f0b4fa6c
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     9  100     9    0     0    121      0 --:--:-- --:--:-- --:--:--   121

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/bash -c mkdir -p /app/{src,build} &&     curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src &&     chown -R cloudron:cloudron /app/src' returned a non-zero code: 2
Failed to build app. See log output above.

[girish]

Oh, it seems the ARG has to be added for each build stage (don't have much experience with these multi-stage builds).

[girish]

Works after that moving the VERSION arg down. Will get it published tomorrow.

[jimcavoli]

@girish Weird. FWIW, I run with DOCKER_BUILDKIT=1 on all the time - https://docs.docker.com/develop/develop-images/build_enhancements/ for more on that particular feature. Glad you got it sorted and we can get this app out there!

[seeker]

What an awesome find

[jimcavoli]

I've updated https://git.cloudron.io/jimcavoli/openslides-app for OpenSlides 3.3

[yusf]

@jimcavoli Oh wow, this with LDAP and large voting sessions can become a breeze.

[jimcavoli]

@yusf I agree. Unfortunately it only supports SAML for SSO so far. Could send a feature request upstream. I've also got a partially-implemented packaging of the Shibboleth IdP for Cloudron, but that's a little stalled while I work out how to deal with some of its idiosyncrasies

[jimcavoli]

@girish Any updates on progress to app store for this one?

[girish]

@jimcavoli I tried building it again the other day and still got the same error. So, I have to investigate why. Maybe something with multi-state builds and our build server.

Step 1/33 : FROM cloudron/base:2.0.0 AS base
 ---> afa4cfc125b4
Step 2/33 : ARG OPENSLIDES_VERSION=3.3
 ---> Using cache
 ---> ab03eb3a365d
Step 3/33 : ARG NODE_VERSION=13.14.0
 ---> Using cache
 ---> fb11ecf942b5
Step 4/33 : RUN apt-get update &&     apt-get -y install       apt-transport-https       bzip2       curl       g++       gcc       git       gnupg2       libpq-dev       make       postgresql-client       rsync       wait-for-it       wget       xz-utils       zlib1g-dev       libffi-dev       dnsutils       iputils-ping       netcat       procps       traceroute       libxml2-dev       libxmlsec1-dev       libxmlsec1-openssl       pkg-config       python3-setuptools       gunicorn3     &&     rm -rf /var/cache/apt /var/lib/apt/lists
 ---> Using cache
 ---> 74dcae334cc2
Step 5/33 : FROM base as build
 ---> 74dcae334cc2
Step 6/33 : RUN mkdir -p /app/{src,build} &&     curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src &&     chown -R cloudron:cloudron /app/src
 ---> Running in e115acee95fb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     9  100     9    0     0     84      0 --:--:-- --:--:-- --:--:--    85

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/bash -c mkdir -p /app/{src,build} &&     curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src &&     chown -R cloudron:cloudron /app/src' returned a non-zero code: 2
Failed to build app. See log output above.

[girish]

@jimcavoli Do you mind if I make it a single stage Dockerfile like the other apps?

edit: Actually, that seems easy since you have already written in to branch off the base nicely.

[jimcavoli]

@girish I could try to adjust it to be a straight-through Dockerfile. Alternately, if you aren't running with DOCKER_BUILDKIT=1 on that machine (which it appears not to be based on output), would you want to give that a shot? As of Docker 20.10.0 (released 8 December 2020), it is considered a stable core feature.

I've got other packagings coming soon that are written to use the same build features, especially when there's a lot of things to build/download, so it'd be nice to know where we're going to land on this. Looks like packages on both my 18.04 and 20.04 boxes are at 19.03.8 which is the 10 March 2020 release. I do happen to be running 20.10.2 on my development machine (and the remote Ubuntu 20.04 build server to be technical, which is the latest, 04 January 2021 release), but it's not like the artifacts are any less compatible after being output with recent versions (those containers ship just fine onto my test Cloudron).

It seems to me like the buildkit engine should be preferable and backward-compatible as described in https://docs.docker.com/develop/develop-images/build_enhancements/ certainly in the 20.x series and beyond, but I've had no issues using it full-time for over a year on the 19.x series and mixing 20.x builds to 19.x runtimes as well.

[girish]

@jimcavoli Yes, I can fix it up. I actually use our build service app which doesn't have BUILDKIT set (assuming, it's off by default). Currently, all cloudrons and all our related services are around Docker 19. I guess we should upgrade Docker to 20 in the next Cloudron release which will help us move forward with the buildkit stuff.

[jimcavoli]

@girish Sounds good; let me know if you need anything along the way - happy to help in any way I can!

[bubonicfred]

@jimcavoli any progress on this, have to run a small conference in a month.

[girish]

@bubonicfred I was just looking into this today. @jimcavoli has finished up the packaging as such.

@jimcavoli where did you find the packaging instructions/docs from? I was trying to just follow through the Dockerfile but it seems quite complicated and I could not find any instructions or code on the openslides site atleast