OpenSlides - digital motion and assembly system
-
@jimcavoli excellent! Hopefully before long we'll be able to have large gatherings again! (not holding my breath though - here in the we UK can't even sort a decent test and trace system despite having blown Ā£12 billion on one!)
-
Happy to announce that this is running on my testbed server presently, and fairly responsively after a slight memory bump (256M just isn't enough to keep all the workers alive, even for one user). Going to try toggling on some of the other optional bits, but then this should be near ready for the
unstablechannel. User management is all in-app for better or worse, but that's an upstream limitation (only supports SAML optionally to act as an external IdP). -
@jimcavoli Very cool. Where is the repo? Also, does the app work well?
-
@girish Works really well; repo forthcoming. I'm still working locally. The main trick just seems to be setting a reasonable floor for the memory limit; I think it's going to be 768M or so as a minimum to ensure the workers all stay up and responsive for a reasonable handful of connections. It makes heavy use of web sockets so it's a hair resource intensive, but given the nature of it, it shouldn't be a big deal for folks to deploy, test, scale up temporarily for an event, and scale back or remove after it's done. The app is rather specifically designed that each installation is one event, something they may change in the future, but for now, really makes running it on Cloudron so much more sensible than any other methodology. (their documentation on how to run it is garbage, basically though; really pushing the hosted version with the dearth of docs)
I'll just finish a little more testing, especially around file uploads and media handling, tonight, and that repo will be linked here following.
-
Repo with initial packaging: https://git.cloudron.io/jimcavoli/openslides-app
I'd regard this as pretty ready to go at this point - all packaging choices were in line with the production configurations recommended by the project. I've tested pretty much everything except SAML manually. User management is independent for the time being as described previously. Also notable is that after some testing, I found
524288000left enough headroom as a memoryLimit floor that the app was usable without constant restarts for light duty (not going to service a 1000 person assembly, probably, but fine for prepping an event and then deciding how to scale up from there). Other notes are in the README of the packaging repo. Also recommend building with buildkit turned on if you don't normally - the JS compilation takes a short eternity, but the Dockerfile takes pretty good advantage of buildkit multistage builds for parallelizing as well as limiting bloat in the final container. -
@jimcavoli awesome, thanks! will take a look next week.
Marked this as wip.
-
@girish Just a bump on this - figure it got lost in the 6.0-releasing shuffle. No huge deal, but hope it can still make
unstablethis year
-
@jimcavoli Not lost
It's marked as https://forum.cloudron.io/tags/wip . But yes, now that we got Cloudron 6 out of the day (save the announcements), we will have time to push these out. -
The build currently fails
Step 6/32 : RUN mkdir -p /app/{src,build} && curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src && chown -R cloudron:cloudron /app/src ---> Running in cb29f0b4fa6c % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9 100 9 0 0 121 0 --:--:-- --:--:-- --:--:-- 121 gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now The command '/bin/bash -c mkdir -p /app/{src,build} && curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src && chown -R cloudron:cloudron /app/src' returned a non-zero code: 2 Failed to build app. See log output above. -
Oh, it seems the ARG has to be added for each build stage (don't have much experience with these multi-stage builds).
-
Works after that moving the VERSION arg down. Will get it published tomorrow.
-
@girish Weird. FWIW, I run with
DOCKER_BUILDKIT=1on all the time - https://docs.docker.com/develop/develop-images/build_enhancements/ for more on that particular feature. Glad you got it sorted and we can get this app out there! -
What an awesome find
-
I've updated https://git.cloudron.io/jimcavoli/openslides-app for OpenSlides 3.3
-
@jimcavoli Oh wow, this with LDAP and large voting sessions can become a breeze.
-
@yusf I agree. Unfortunately it only supports SAML for SSO so far. Could send a feature request upstream. I've also got a partially-implemented packaging of the Shibboleth IdP for Cloudron, but that's a little stalled while I work out how to deal with some of its idiosyncrasies
-
@girish Any updates on progress to app store for this one?
-
@jimcavoli I tried building it again the other day and still got the same error. So, I have to investigate why. Maybe something with multi-state builds and our build server.
Step 1/33 : FROM cloudron/base:2.0.0 AS base ---> afa4cfc125b4 Step 2/33 : ARG OPENSLIDES_VERSION=3.3 ---> Using cache ---> ab03eb3a365d Step 3/33 : ARG NODE_VERSION=13.14.0 ---> Using cache ---> fb11ecf942b5 Step 4/33 : RUN apt-get update && apt-get -y install apt-transport-https bzip2 curl g++ gcc git gnupg2 libpq-dev make postgresql-client rsync wait-for-it wget xz-utils zlib1g-dev libffi-dev dnsutils iputils-ping netcat procps traceroute libxml2-dev libxmlsec1-dev libxmlsec1-openssl pkg-config python3-setuptools gunicorn3 && rm -rf /var/cache/apt /var/lib/apt/lists ---> Using cache ---> 74dcae334cc2 Step 5/33 : FROM base as build ---> 74dcae334cc2 Step 6/33 : RUN mkdir -p /app/{src,build} && curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src && chown -R cloudron:cloudron /app/src ---> Running in e115acee95fb % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9 100 9 0 0 84 0 --:--:-- --:--:-- --:--:-- 85 gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now The command '/bin/bash -c mkdir -p /app/{src,build} && curl -L https://github.com/OpenSlides/OpenSlides/archive/${OPENSLIDES_VERSION}.tar.gz | tar zxf - --strip-components 1 -C /app/src && chown -R cloudron:cloudron /app/src' returned a non-zero code: 2 Failed to build app. See log output above. -
@jimcavoli Do you mind if I make it a single stage Dockerfile like the other apps?
edit: Actually, that seems easy since you have already written in to branch off the base nicely.
-
@girish I could try to adjust it to be a straight-through Dockerfile. Alternately, if you aren't running with
DOCKER_BUILDKIT=1on that machine (which it appears not to be based on output), would you want to give that a shot? As of Docker 20.10.0 (released 8 December 2020), it is considered a stable core feature.I've got other packagings coming soon that are written to use the same build features, especially when there's a lot of things to build/download, so it'd be nice to know where we're going to land on this. Looks like packages on both my 18.04 and 20.04 boxes are at 19.03.8 which is the 10 March 2020 release. I do happen to be running 20.10.2 on my development machine (and the remote Ubuntu 20.04 build server to be technical, which is the latest, 04 January 2021 release), but it's not like the artifacts are any less compatible after being output with recent versions (those containers ship just fine onto my test Cloudron).
It seems to me like the buildkit engine should be preferable and backward-compatible as described in https://docs.docker.com/develop/develop-images/build_enhancements/ certainly in the 20.x series and beyond, but I've had no issues using it full-time for over a year on the 19.x series and mixing 20.x builds to 19.x runtimes as well.
-
@jimcavoli Yes, I can fix it up. I actually use our build service app which doesn't have BUILDKIT set (assuming, it's off by default). Currently, all cloudrons and all our related services are around Docker 19. I guess we should upgrade Docker to 20 in the next Cloudron release which will help us move forward with the buildkit stuff.
-
@girish Sounds good; let me know if you need anything along the way - happy to help in any way I can!
-
@jimcavoli any progress on this, have to run a small conference in a month.
-
@bubonicfred I was just looking into this today. @jimcavoli has finished up the packaging as such.
@jimcavoli where did you find the packaging instructions/docs from? I was trying to just follow through the Dockerfile but it seems quite complicated and I could not find any instructions or code on the openslides site atleast
-
@girish said in OpenSlides - digital motion and assembly system:
I was trying to just follow through the Dockerfile but it seems quite complicated and I could not find any instructions or code on the openslides site atleast
Is one of these what you're looking for?
https://github.com/OpenSlides/openslides-docker-compose
https://github.com/OpenSlides/OpenSlides/tree/master/docker
https://github.com/OpenSlides/OpenSlides/blob/master/README.rst#setup-docker-images -
@girish Yeah, what @jdaviescoates posted is the thing - it's largely not documented all that well from the technical setup perspective. I just sort of...figured it out? There's not that many components and it's a pretty standard python app. Because it's not well-documented on the project's part insofar as running is concerned, that's a large part of why I made the readme and put all the notes in that I did.
-
@jimcavoli Oh, I completely missed your notes in the README ! I will take a look again.
-
I've updated https://git.cloudron.io/jimcavoli/openslides-app with the latest changes for version 3.3 of Openslides. Dockerfile is still multi-staged for the time being. @girish do you want to take another pass at building as-is or should I work on making the Dockerfile straight through.
-
@jimcavoli I think Dockerfile straight would be great. How much work do you think it is? Maybe you can quickly try hacking away and see if it's not too much work?
-
@girish So the gzip failure is independent the staged builds. Arguing with tar/gzip and GitHub about file formats presently, but I can replicate the issue you were having and am testing against the same Dockerfile with and without buildit on, so once I get that sorted, we can regroup and go from there.
-
@jimcavoli said in OpenSlides - digital motion and assembly system:
I've updated https://git.cloudron.io/jimcavoli/openslides-app with the latest changes for version 3.3 of Openslides. Dockerfile is still multi-staged for the time being. @girish do you want to take another pass at building as-is or should I work on making the Dockerfile straight through.
Hi, Openslides isn't actually in the cloudron-appstore - is it broken?
-
@hollosch No, work remains "in progress" for the time being to get a reliable package finished before it heads there. You can keep track by the "WIP" tag on the thread right now - it'll go "Solved" and green once completed
-
Hi @girish do you think Openslides can work on Cloudron ? Any news about it ?
-
@Benoit have to investigate, it's quite a complex app to package ! See https://github.com/OpenSlides/OpenSlides#development . Last, I checked, there were no clear instructions for installations from source, so one has to reverse engineer from their docker files.
-
@girish Iām still around - I can pick this back up in the next week or two
-
@jimcavoli said in OpenSlides - digital motion and assembly system:
@girish Iām still around - I can pick this back up in the next week or two
Be my guest!
-
@jimcavoli yes, please
I will be happy to get this integrated. I think I have addressed the platform issues in the releases since. -
@girish @jimcavoli
Any movement on this going into the app store?
