Cloudron on a Raspberry pi?
-
@malvim We never run tests on the Cloudron itself, only on the laptop! Can you give the output of the
npm testcommand? What is the error? -
@malvim Well, consider yourself not alone in this endeavor! I'd love to help and am very interested in porting this to arm. Depends on the number of dependencies from the base image as well as Cloudron itself.
I'll post back here when I get my Raspberry Pi; I'm excited!
️ -
@girish said in Cloudron on a Raspberry pi?:
@malvim We never run tests on the Cloudron itself, only on the laptop! Can you give the output of the
npm testcommand? What is the error?malvim@zem:~/docker/postgresql-addon/test$ npm i audited 152 packages in 2.419s found 3 vulnerabilities (2 low, 1 high) run `npm audit fix` to fix them, or `npm audit` for details malvim@zem:~/docker/postgresql-addon/test$ npm test > postgresql-addon@1.0.0 test /home/malvim/docker/postgresql-addon > mocha --bail ./test/test.js Postgresql Addon Error: No such container: postgresql auth ✓ fails without access_token ✓ fails with invalid access_token ✓ succeeds add database ✓ succeeds (676ms) ✓ succeeds when added again (124ms) remove database 1) succeeds 5 passing (2m) 1 failing 1) Postgresql Addon remove database succeeds: Error: Timeout of 100000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/malvim/docker/postgresql-addon/test/test.js) at listOnTimeout (internal/timers.js:554:17) at processTimers (internal/timers.js:497:7)Hey, @girish!
This is the output, and it just hangs after that and never exits. This happens on my laptop and on another machine I used for testing purposes. On my Cloudron, these tests passed and only the last one failed, the backup/restore one iirc.
-
@malvim said in Cloudron on a Raspberry pi?:
Postgresql Addon
Error: No such container: postgresql
auththis seems to be more of a problem than the timeout later
-
@robi said in Cloudron on a Raspberry pi?:
@malvim said in Cloudron on a Raspberry pi?:
Postgresql Addon
Error: No such container: postgresql
auththis seems to be more of a problem than the timeout later
It's really not. This happens because the test always tries to remove the currentl running postgresql container before running the tests, so the first time you run, the container is not running and you get this message. Still runs the tests and some of them work.
-
@malvim ok thanks, otherwise it looks like it's missing
..which would partially explain the timeout..
-
@malvim said in Cloudron on a Raspberry pi?:
This is the output, and it just hangs after that and never exits
Is there anything in
docker logs -f postgresql? -
Is it getting the token last when it should be getting it first in
auth? I've gotta re-read this whole thread
cause I'm missing what this unit test even is supposed to prove. That one module could run in a Cloudron environment? -
@girish said in Cloudron on a Raspberry pi?:
Is there anything in
docker logs -f postgresql?Nothing that I thought was strange... Here's the output:
Creating new installation [39/54] The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "C". The default text search configuration will be set to "english". Data page checksums are disabled. fixing permissions on existing directory /var/lib/postgresql/11/main ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting default timezone ... Etc/UTC selecting dynamic shared memory implementation ... posix creating configuration files ... ok running bootstrap script ... ok performing post-bootstrap initialization ... ok syncing data to disk ... ok WARNING: enabling "trust" authentication for local connections You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. Success. You can now start the database server using: /usr/lib/postgresql/11/bin/pg_ctl -D /var/lib/postgresql/11/main -l logfile start CREATE ROLE ALTER ROLE waiting for server to shut down.... done server stopped Generating SSL certificate Generating a RSA private key .......................+++++ .....................................................................+++++ writing new private key to '/run/postgresql.cloudron.key' ----- Starting supervisor 2020-10-21 01:20:33,612 CRIT Supervisor running as root (no user in config file) 2020-10-21 01:20:33,613 INFO Included extra file "/etc/supervisor/conf.d/postgresql-service.conf" during parsing 2020-10-21 01:20:33,613 INFO Included extra file "/etc/supervisor/conf.d/postgresql.conf" during parsing 2020-10-21 01:20:33,628 INFO RPC interface 'supervisor' initialized 2020-10-21 01:20:33,628 CRIT Server 'inet_http_server' running without any HTTP authentication checking 2020-10-21 01:20:33,630 INFO RPC interface 'supervisor' initialized 2020-10-21 01:20:33,631 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2020-10-21 01:20:33,631 INFO supervisord started with pid 1 2020-10-21 01:20:34,635 INFO spawned: 'postgresql' with pid 53 2020-10-21 01:20:34,639 INFO spawned: 'postgresql-service' with pid 54 2020-10-21 01:20:34.680 UTC [53] LOG: listening on IPv4 address "0.0.0.0", port 5432 2020-10-21 01:20:34.680 UTC [53] LOG: listening on IPv6 address "::", port 5432 2020-10-21 01:20:34.714 UTC [53] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2020-10-21 01:20:34.788 UTC [61] LOG: database system was shut down at 2020-10-21 01:20:33 UTC 2020-10-21 01:20:34.825 UTC [53] LOG: database system is ready to accept connections Postgresql service endpoint listening on https://:::3000 [GET] /healthcheck 2020-10-21 01:20:36,382 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-10-21 01:20:36,382 INFO success: postgresql-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) [GET] /healthcheck [GET] / [GET] / [GET] / [POST] /databases [POST] /databases [POST] /databases [DELETE] /databases/removetestdatabase -
So, could anyone here just try and run tests for addons on their machines, without any cloudron-related stuff, then just clone an addon and try to test it?
I've ran tests on my laptop, and on a server I have access to that doesn't run cloudron, and both of them time out in the same place.
DON'T RUN this tests on your production cloudron like I did, it will delete and recreate your postgresql (or whatever addon you're trying to use) and apps will go down heheh. I had to restore a couple of apps' backups, but it's all good now.
-
@malvim Maybe you can try to see if the mysql addon tests work in the meantime?
I ran the postgresl tests now and it did work for me:
> postgresql-addon@1.0.0 test /home/girish/yellowtent/postgresql-addon > mocha --bail ./test/test.js Postgresql Addon Error: No such container: postgresql Error response from daemon: network with name cloudron already exists auth ✓ fails without access_token ✓ fails with invalid access_token ✓ succeeds add database ✓ succeeds (401ms) ✓ succeeds when added again remove database ✓ succeeds (147ms) use the database ✓ can create extension (77ms) ✓ can create table foo ✓ can insert into table foo ✓ can read from table foo ✓ restart (5309ms) ✓ can read from table foo backup and restore ✓ succeeds to create backup (386ms) ✓ succeeds to create new database (743ms) ✓ succeeds to clear new database (439ms) ✓ succeeds to restore backup (1280ms) ✓ succeeds to check restore data (44ms) restore of invalid dump fails ✓ succeeds to create backup (178ms) ✓ succeeds to clear new database (426ms) ✓ fails to restore backup (180ms) restore of existing dump ✓ succeeds (1876ms) 21 passing (38s) -
Hey, @girish
Mysql addon tests run perfectly on all three machines (laptop, server, pi)!
I'm having failing tests on mail and sftp addons, and this hanging problem with postgresql. All behaviors are the same on the three machines, which is a... good thing, I guess? Haha! I'll try the others and see where they go.
Couldn't download graphite, though, says I don't have permission rights.
-
@malvim The mail tests are kind of hard to run, they also require a special setup with DNS/Docker. We can skip that for now. Mongo and redis tests work too? That's really good progress then. Can you tell me what you face with the sftp addon? You might need the latest "build" of sftp since I really just fixed the test 2-3 days ago.
-
Hey,
It's a bit past midnight here and I've JUST realized my laptop doesn't have ssh HOST keys (I never ssh into it), and the test mounts the host's
/etc/sshdirectory into the container's/etc/ssh, and that's why it's not running.I'll go get some sleep and keep going tomorrow night.
-
So I installed
openssh-serverand now I get this:2020-10-22 04:13:28,130 INFO spawned: 'proftpd' with pid 48 Wrong passphrase for this key. Please try again. Wrong passphrase for this key. Please try again. Wrong passphrase for this key. Please try again. 2020-10-22 04:13:28,150 c7a6e160fe35 proftpd[48] c7a6e160fe35: mod_sftp/0.9.9: error reading passphrase for SFTPHostKey '/etc/ssh/ssh_host_rsa_key': (unknown) 2020-10-22 04:13:28,150 c7a6e160fe35 proftpd[48] c7a6e160fe35: mod_sftp/0.9.9: unable to use key in SFTPHostKey '/etc/ssh/ssh_host_rsa_key', exiting 2020-10-22 04:13:28,151 INFO exited: proftpd (exit status 0; not expected) 2020-10-22 04:13:31,160 INFO spawned: 'proftpd' with pid 49 Wrong passphrase for this key. Please try again. Wrong passphrase for this key. Please try again. Wrong passphrase for this key. Please try again. 2020-10-22 04:13:31,183 c7a6e160fe35 proftpd[49] c7a6e160fe35: mod_sftp/0.9.9: error reading passphrase for SFTPHostKey '/etc/ssh/ssh_host_rsa_key': (unknown) 2020-10-22 04:13:31,183 c7a6e160fe35 proftpd[49] c7a6e160fe35: mod_sftp/0.9.9: unable to use key in SFTPHostKey '/etc/ssh/ssh_host_rsa_key', exiting 2020-10-22 04:13:31,185 INFO exited: proftpd (exit status 0; not expected)Not sure how to proceed when it asks me for passphrases...
-
are there pre-set keys or are they generated during setup?
since the dir wasn't there during setup, maybe the generation failed hence the errors.
-
@malvim I guess you are running this on Ubuntu 20? They changed the ssh keys format, so you have to generate the host keys again. See the test.js comment in startSftp().
-
@robi setup mounts the host's
/etc/sshdir, so it uses whatever is in the host machine, and it was... nothing!@girish hahahahahahahaha holy crap, I was looking at THIS EXACT LINE and completely disregarded the comment that explained perfectly what was going on! Thanks for the help once again, I'll check it out.
-
Hey, guys.
So, this is what's going on currently:
I was able to build and run tests for, with minimal adaptations, the
docker-base-imageproject, and the following addons:docker-sftpmongodb-addonmysql-addonredis-addonturn-addon
That leaves
mail-addon,docker-graphiteandpostgresql-addonMail needs more setup as @girish said, so I'm not doing it for now. I was not able to clone the
docker-graphiteproject, as I think I've mentioned before, so still waiting on what to do in this case.In trying to understand a bit more of what's going on with the
postgreql-addontests hanging, I found it hangs in any test that usesconnectClient, like theremove databaseanduse databasetests:describe('remove database', function () { const data = { database: 'removetestdatabase', username: 'removetestuser', password: 'somepassword', locale: 'C' }; before(function (done) { addDatabase(data, done); }); it('succeeds', function (done) { request.delete(`https://${ip}:3000/databases/${data.database}?access_token=${CLOUDRON_POSTGRESQL_TOKEN}&username=${data.username}`, { rejectUnauthorized: false }, function (error, response, body) { expect(error).to.be(null); expect(response.statusCode).to.equal(200); connectClient(data, function (error) { expect(error).to.not.eql(null); client.end(done); }); }); }); });In this case,
addDatabaseruns ok, and it makes sense since it is tested before.The function is called,
request.deleteis called, the twoexpects pass, and then the function INSIDEconnectClient(with theexpect(error)line) never runs.Can anyone else (besides @girish, who ran the tests and seen them run fine) run this and see what happens? This is happening on my laptop as well, not only on the Pi, but if I can't get this to go, I can't get the tests to pass on the pi, and we won't be able to trust everything runs okay.
-
@malvim Are you testing this on a mac?
-
@girish Nope. Ubuntu 20.04 on both my laptop and a server, Ubuntu 18.04 on the raspberry pi. All with the same result.
-
It works for me here atleast. I am guessing that you are unable to connect to the container IP maybe? Can you try installing psql tooling and connect via IP address? You can put a log in connectClient to see the credentials.
$ npm test > postgresql-addon@1.0.0 test /home/girish/yellowtent/postgresql-addon > mocha --bail ./test/test.js Postgresql Addon Error response from daemon: network with name cloudron already exists auth ✓ fails without access_token ✓ fails with invalid access_token ✓ succeeds add database ✓ succeeds (369ms) ✓ succeeds when added again remove database ✓ succeeds (125ms) use the database ✓ can create extension (82ms) ✓ can create table foo ✓ can insert into table foo ✓ can read from table foo ✓ restart (5091ms) ✓ can read from table foo backup and restore ✓ succeeds to create backup (443ms) ✓ succeeds to create new database (412ms) ✓ succeeds to clear new database (426ms) ✓ succeeds to restore backup (940ms) ✓ succeeds to check restore data restore of invalid dump fails ✓ succeeds to create backup (186ms) ✓ succeeds to clear new database (424ms) ✓ fails to restore backup (185ms) restore of existing dump ✓ succeeds (1721ms) 21 passing (36s) -
Hey,
So I commented out the
remove databasetests because I had trouble connecting from the container, but the next test,use the database, starts like this:// psql -h localhost -U usetestuser -d usetestdatabase describe('use the database', function () { const data = { database: 'usetestdatabase', username: 'usetestuser', password: 'somepassword', locale: 'C' }; before(function (done) { async.series([ addDatabase.bind(null, data), connectClient.bind(null, data) ], done); });So I ran "psql -h localhost -U usetestuser -d usetestdatabase", using 'somepassword' as a password, and was able to connect, both from inside the container and from my host machine.
connectClientstill doesn't connect, and, I get this on the screen:auth ✓ fails without access_token ✓ fails with invalid access_token ✓ succeeds use the database usetestuser somepassword usetestdatabase 172.18.0.2 5432 1) "before all" hook 3 passing (2m) 1 failing 1) Postgresql Addon use the database "before all" hook: Error: Timeout of 100000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/malvim/Projetos/Pi/docker/postgresql-addon/test/test.js) at listOnTimeout (internal/timers.js:551:17) at processTimers (internal/timers.js:494:7) 2) "after all" hook -
Also, it does not seem to be network-related, since I get the same behavior on an ubuntu 20.04 VPS I have...
-
I tried recreating the steps inside the container using node, and could connect. I'm really at a loss here right now...
-
@malvim Wish I had my Pi now so I could help test! Soon tho.
️ -
Hey, @Lonk! You mind testing this on your laptop? I'm not getting these errors only on the Pi, I'm getting them on my amd64 laptop and amd64 server as well!
Repo is here:
https://git.cloudron.io/cloudron/postgresql-addon, it's just a matter of:- cloning it;
- building the image locally (
docker build -t cloudron/postgresqladdontest .) - installing dependencies (
npm install) - running the test (
npm test)
Could you run this somewhere and see if you have the same problem?
Thanks!
-
MAN, it seems I'm running into ALL KINDS of weird problems hahaha!
I started going into postgresql node client's code, and it seemed to be some weird behavior of
EventEmitter, which was weird...I installed
nvmand tested with older node distributions, and it ran fine on my laptop! So it seems, @girish, that the test code forpostgresql-addondoes not run with node v14 or later on my machines. Tested with v12 and v13 and it was all fine. What version are you using for tests?Moving on to the next hurdle, I guess hahah!
-
@malvim said in Cloudron on a Raspberry pi?:
So it seems, @girish, that the test code for postgresql-addon does not run with node v14 or later on my machines
Aha! I feel like I have hit this issue before. Indeed, when we updated the box code to use node 14 lots of things fail (not sure why). We use node 10.18.1 everywhere. This is why the hotfix and code enforces it. For future reference, always use the node in
scripts/createReleaseTarballin the box repo. -
@girish okay, cool, I'll use that. It's good news, then, that the OTHER tests even ran!
So, for what it's worth, the code seemed to fail on the use of
EventEmitter. A class was dispatching an event and the corresponding listener was not triggered, so if you ever need to investigate further, maybe that's a start. -
I have ordered a RaspberryPi 4 now as well and plan to use that as a home server, so hopefully we can get this supported well in the future
Very curious to get my hands on what you have already managed to get working! -
There's the new keyboard that includes an rPi inside too.
-
@malvim I've also got a Raspberry Pi 4 now. Can you give an update on where you are and do you have forked repos with arm changes somewhere?
-
To give a short update from my side, with the information already posted here, I was able to get the box (the main Cloudron controller process) up and running on the Pi 4 as well as successfully install Cloudron as such. I have only just started on the base image and the other addons, so any patches here are welcome.
To collect the changes, I am creating
arm64branches in the relevant repos, for example https://git.cloudron.io/cloudron/box/-/commits/arm64 and https://git.cloudron.io/cloudron/docker-base-image/-/commits/arm64On top of this, I am trying to implement a better provisioning workflow for development, this is similar to the hotfix, so it is still aimed towards developers porting stuff to arm. More info on this later.
-
@nebulon So quickly! Nice work.
I want to convert my app to run on ARM so I'll be getting the same board you got to verify the OpenVPN Client and all of its features are able to work in an ARM environment.
This is so cool, love ARM.
-
Well it really remains to be seen how powerful such a board is to run common apps through docker
-
@nebulon True, but even if it could run 1. Not even that well, imagine a future with the Rasberry Pi 8. ARM CPUs are getting insane.
-
@nebulon said in Cloudron on a Raspberry pi?:
Well it really remains to be seen how powerful such a board is to run common apps through docker
I'm running like 4 containers on my Raspberry pi at home, it's super smooth, and it's only a RPi 2 ! The RPi 4 is gonna be more than capable of running a few apps for home usage I think
-
@nebulon Great work!
I got really caught up with work and personal stuff over the last weeks, so I was not able to keep on working.
On most addon repos, I was creating
arm64branches as well, but most of them were just a matter of changing the base image to not have the hash. I was usingcloudron/baseimageand building it on the pi itself before trying to install cloudron, so I had it tagged locally with that name, and others wouldn't download from docker hub, using the local arm64 one instead.I'll search for any changes I have made over here, but they're not a lot.
I'll post a suymmary tonight.
-
@malvim right, it was same route for me then. I have just managed to get all addons and nextcloud to run. The notes here about mongodb were helpful!
Overall I don't think it will be included in the next release though, maybe as something experimental, but while we have a proof-of-concept now, it will still take quite some time to actually make it proper and of course all apps have to be rebuilt...
-
@nebulon Let me know when you send are finished with your branch (and how we'd build it differently) and I'll make sure my OpenVPN Client is ready!
-
@lonk Can you make a new post for the VPN Client with the current status? I would like to discuss a bit whether it should be part of the Cloudron box code or an app.
-
@girish On it.
-
@malvim / @nebulon - I just picked up a Raspberry Pi, I'm going to start working on this myself. Have either of you gotten any further since your last posts in Nov?
-
@lonk I basically got this working already, there are only a few changes required to make the base system work. That is very minor and has more to do with Ubuntu setup rather than our code base.
However the main reason we have not pursued this further is, that in order to support arm (arm64 to be precise), we have to rebuild all addon docker images for a start and patch up the code which creates addon container accordingly with different image tags and even once that is done, we then have to rebuild all app packages for arm as well, which means a lot of testing and potentially fixing apps upstream. This is a lot of work and this has to be done and tested for every app package update of course.
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I do think it is worth it in the long run to support arm, also because VPS provider start adding arm options and at least the raspberrypi 400 showed okish performance while I was doing the proof-of-concept, however we will need a different way to build packages and run the selenium tests for both architectures reliably. This is currently done manually by us due to the lack of such CI/CD pipelines in place.
So all this is certainly doable but unless we see higher demand, it is hard to justify the extra work and for the time being essentially at least double the work per app update.
-
@nebulon Well, that sounds to me like like not much work needs to be done, I'll do some PoC stuff myself and see the difficulty of converting the app. Docker is supposed to support multi arch so it's a matter of getting all the developer's to support multi arch upstream. So, I'll start submitting tickets for those things. Thanks for telling me where you were at, do you have your POC up anywhere yet?
-
@lonk for the core box code, essentially only two things were needed. The branch is at https://git.cloudron.io/cloudron/box/-/commits/arm64/
-
@nebulon said in Cloudron on a Raspberry pi?:
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I have some experience with this and have set up my own multi-arch go build pipelines using a single Dockerfile for some of my other apps: minitor-go, dockron, tag-checker, and for Python ones too: original minitor.
Here's a sample repo demonstrating my process: multiarch-pipeline-test. It's easier these days if your server has
docker buildxthough.Also, since with Cloudron we're most often building things that exist upstream, here's an example multi-arch build repo I have for the Golang project cadvisor. It will auto build a particular cadvisor version on a git tag so I just need to create a release on my Gitea server and the build is started and deployed. With cadvisor, I have to clone the whole repo and cross-compile the cadvisor binary for arm becaue there is no pre-compiled binary. If there is, it should be even easier to just pull that binary.
Anyway, I'm happy to help if there are any applications that may be critical to be ported.
-
@iamthefij nice thanks for the pointers, when we got started on this this will help. I tried to use
docker buildxbut I couldn't get it to work and produce binaries which would actually run on the raspberrypi, so I ended up using the raspberrypi to build the images itself, which surprisingly showed how beefy that board has become -
@nebulon careful with compiling on rPi's as they can overheat and burn out
Having a small heatsink or fan handy helps a lot.Life of Gratitude.
Life of CT Gifts -
@robi yup, I have a case for it with cooling
-
@nebulon You have that case that is the heatsink? It’s very nice.
-
@nebulon yup! I tried it as well, and could not get it to work. building on the pi itself proved to be
the easiest way... -
@yusf yes exactly that, makes it feel like a strong brick you could throw around
-
@nebulon would all binaries not run (eg.
/bin/shfrom within the base) or just Go binaries that you compiled within thebuildxpipeline? If it's the former, it may not be using the right base image. If it's the latter and the former works, perhaps setting theGOARCHvariable via a build-arg would solve it.Note: I personally have not used
buildxyet, but from what I can see it's a simpler, automatic version of what I'm trying to do with qemu that handles the manifest for you. So I think you should just be able to build without the muckiness of all the build-args I pass, but if not you can play with mixing those in until it works.I think
buildxis supported on my laptop, so I can give it a try, but it's not supported on my CI box yet, so I haven't switched.
