Cloudron on a Raspberry pi?

[malvim]

@girish said in Cloudron on a Raspberry pi?:

I'm in the middle of running installation without unbound, and everything worked. I'll re-run it now with unbound again, but host my.pi.domain.com does NOT work from any machine inside my home network, and does work from my main cloudron server, so I think there's more to debug here, unfortunately.

host my.pi.domain.com DOES work from my local machine, but not using Google's server, which is kind of weird.

I guess I'll have to solve this issue (maybe with my ISP?) before proceeding with cloudron on the pi. Sucks.

[malvim]

Okay, so this was related to my ISP. After a few calls with tech support and buying a new router, DNS issues are gone and I was able to install cloudron from start to finish!

Now I ssh into the pi and, as expected, the first containers (mysql, turn, sftp, graphite) are continuosly restarting, since they're installed from the production amd64 images which won't work.

So now I'm thinking I'll just clone, say, the mysql addon (from https://git.cloudron.io/cloudron/mysql-addon.git), try to build it for the pi, publish the image on my repo and try to use it, see if mysql works, and go from there. I'll try to do that tonight and get back to you guys.

What do you guys say? @girish ? Is that the right path to start on?

Baby steps.

[girish]

@malvim w00t, awesome progress. If you can create a branch of your box changes, that will help others as well. Yes, starting with one of the addon containers is a good start. They all have automatic tests, it's easy to run them as well.

[malvim]

Ah, just noticed I might need a new cloudron baseimage, right? Hahah! This is gonna be fun

[malvim]

Hey,

So I'm building docker-base-image and the other initial docker containers, but I can't seem to find them on cloudron's git repo.

I found this code in infra_version.js:

    'images': {
        'turn': { repo: 'cloudron/turn', tag: 'cloudron/turn:1.1.0@sha256:e1dd22aa6eef5beb7339834b200a8bb787ffc2264ce11139857a054108fefb4f' },
        'mysql': { repo: 'cloudron/mysql', tag: 'cloudron/mysql:2.3.1@sha256:c1145d43c8a912fe6f5a5629a4052454a4aa6f23391c1efbffeec9d12d72a256' },
        'postgresql': { repo: 'cloudron/postgresql', tag: 'cloudron/postgresql:3.1.0@sha256:261c38d332a20cd4160930d7395fd342496159e94c522d92fde8163c680adc98' },
        'mongodb': { repo: 'cloudron/mongodb', tag: 'cloudron/mongodb:3.0.0@sha256:59e50b1f55e433ffdf6d678f8c658812b4119f631db8325572a52ee40d3bc562' },
        'redis': { repo: 'cloudron/redis', tag: 'cloudron/redis:2.3.0@sha256:0e31ec817e235b1814c04af97b1e7cf0053384aca2569570ce92bef0d95e94d2' },
        'mail': { repo: 'cloudron/mail', tag: 'cloudron/mail:2.10.0@sha256:3aff92bfc85d6ca3cc6fc381c8a89625d2af95cc55ed2db692ef4e483e600372' },
        'graphite': { repo: 'cloudron/graphite', tag: 'cloudron/graphite:2.3.0@sha256:b7bc1ca4f4d0603a01369a689129aa273a938ce195fe43d00d42f4f2d5212f50' },
        'sftp': { repo: 'cloudron/sftp', tag: 'cloudron/sftp:2.0.2@sha256:cbd604eaa970c99ba5c4c2e7984929668e05de824172f880e8c576b2fb7c976d' }
    }

And I could find the database addons (mysql-addon. postgresql-addon and mongodb-addon), but I'm not sure they're what you use to build these images, and also I wasn't able to find anything to do with turn, redis, sftp, graphite...

I was able to build mysql-addon on top of my arm64 base image, but can't seem to find the others in order to keep going. Are these Dockerfiles somewhere else we have access to?

[mehdi]

@malvim said in Cloudron on a Raspberry pi?:

Thanks for the replies, guys. I'll take a stab at it, tomorrow or on Friday, and see what it looks like.

I honestly had about 0 confidence that we could go from there to :

@malvim said in Cloudron on a Raspberry pi?:

I was able to install cloudron from start to finish!

in like a month...

Hat's off

[nebulon]

The mysql-addon repo is at https://git.cloudron.io/cloudron/mysql-addon and according to that repo naming convention also the others.

The sftp addon is at https://git.cloudron.io/cloudron/docker-sftp (not sure why that repo name ended up like it is)

On a side topic, does anyone know of some good naming convention for docker images when it comes to supporting multi-arch?

For example:
cloudron/mysql-amd64:2.3.1 (amd64)
cloudron/mysql-arm64:2.3.1 (arm64)

[fbartels]

@nebulon said in Cloudron on a Raspberry pi?:

On a side topic, does anyone know of some good naming convention for docker images when it comes to supporting multi-arch?

I think the preferred way is to not have the architecture in the name or tag, but rather populate the manifest properly: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/

[girish]

@malvim we build images manually using docker build and push them out. When building you have to tag it locally as cloudron/mysqladdontest (just docker build -t cloudron/mysqladdontest.). Do the tests work?

[malvim]

@nebulon thanks! I was able to find most of them, but still missing graphite, turn and mail. Are those regular docker containers as well, and if so, are their dockerfiles published on git.cloudron? A search for those terms did not yield any meaningfule results for me.

On your side topic: I went looking for the arm64 version of the base ubuntu image, and it's published as multi-platform, so I guess there's no need to publish under another name, like @fbartels already answered. What I did have to do, though, was to strip the sha256 hash from the FROM statement. When I tried building on the rpi keeping the hash, docker selected the amd64 version of the image. Not sure how we'd go about making sure the image passes the integrity check (which I assume is why the has is there in the first place) while also being able to build for different architectures.

@mehdi Man, I appreciate it! It was almost about a month just to get my hands on one, then weird networking stuff, trying to sneak in a few hours of banging on this problem in between work... Haha! A lot of fun though! So thanks for the kudos, they're highly appreciated, especially coming from you!

@girish yeah, I'm publishing to a private docker container (which I packaged together with verdaccio in a custom app just to have docker and npm private registers haha), so that part is good, I'm tagging them pointing to it and it's all good.

Haven't run the tests yet, though.

[malvim]

Okay, so another question: Do we depend on mongodb being 4.0, or can we upgrade it to 4.2?

It seems mongodb 4.0 on arm64 only has support for ubuntu 16.04 (as per https://docs.mongodb.com/manual/administration/production-notes/)

[girish]

https://git.cloudron.io/cloudron/turn-addon is the turn addon and https://git.cloudron.io/cloudron/docker-graphite/ is graphite (the names are a bit here and there). The mail server is not open, I have sent you an invite though.

[malvim]

@girish Thanks, I'll check them out!

[girish]

@malvim said in Cloudron on a Raspberry pi?:

Okay, so another question: Do we depend on mongodb being 4.0, or can we upgrade it to 4.2?

yeah, we can bump it. But it requires the usual round of testing against all apps.

[malvim]

@girish Cool. I'll bump it to 4.2 (4.4 needs ubuntu 20.04) and keep pushing hehehe. I'll ping people here when I have something we can run tests on.

[girish]

@malvim Are you using buildx? We bumped 5.6.2 to have docker 19 so we can have buildx

[malvim]

@girish Not yet, I tried installing it for cross-platform building, but had some errors and didn't want to waste time, so I'm now just building the images from the pi itself. Later today I might have to start using it again for installing cloudron, so that might become enought of a hassle that I try again.

Anyone here with experience building for other architectures using buildx?

[malvim]

Hey, all.

Happy to say this is now going on on my raspberry pi:

I have NO idea wheter stuff is really working hahaha.
I'll probably choose a simple app and build it for arm64, then try to install it from command line and run tests, maybe?

Not sure how to run tests against addons, or even apps. Is there some documentation around about this? @girish, could you point me somewhere? Thanks!

[girish]

@malvim Wow, that's some incredible progress! If the status indicator is green, it's pretty sure that the addon containers are responding to health checks!

For the test for the addons, there is a test/ inside the repo of each addon. You can just do npm install and npm test.

[girish]

To be clearer, like this (say with postgres addon). The tests will always test the latest cloudron/{addonname}test image.

$ docker build -t cloudron/postgresqladdontest .
$ cd test
$ npm install # only have to do this once
$ npm test

> postgresql-addon@1.0.0 test /home/girish/yellowtent/postgresql-addon
> mocha --bail ./test/test.js



  Postgresql Addon
Error response from daemon: network with name cloudron already exists
    auth
      ✓ fails without access_token
      ✓ fails with invalid access_token
      ✓ succeeds
    add database
      ✓ succeeds (410ms)
      ✓ succeeds when added again
    remove database
      ✓ succeeds (140ms)
    use the database
      ✓ can create extension (49ms)
      ✓ can create table foo
      ✓ can insert into table foo
      ✓ can read from table foo
      ✓ restart (5241ms)
      ✓ can read from table foo
    backup and restore
      ✓ succeeds to create backup (392ms)
      ✓ succeeds to create new database (422ms)
      ✓ succeeds to clear new database (449ms)
      ✓ succeeds to restore backup (875ms)
      ✓ succeeds to check restore data (47ms)
    restore of invalid dump fails
      ✓ succeeds to create backup (175ms)
      ✓ succeeds to clear new database (434ms)
      ✓ fails to restore backup (174ms)
    restore of existing dump
      ✓ succeeds (1895ms)


  21 passing (35s)