-
Well it really remains to be seen how powerful such a board is to run common apps through docker
@nebulon said in Cloudron on a Raspberry pi?:
Well it really remains to be seen how powerful such a board is to run common apps through docker
I'm running like 4 containers on my Raspberry pi at home, it's super smooth, and it's only a RPi 2 ! The RPi 4 is gonna be more than capable of running a few apps for home usage I think
-
To give a short update from my side, with the information already posted here, I was able to get the box (the main Cloudron controller process) up and running on the Pi 4 as well as successfully install Cloudron as such. I have only just started on the base image and the other addons, so any patches here are welcome.
To collect the changes, I am creating
arm64branches in the relevant repos, for example https://git.cloudron.io/cloudron/box/-/commits/arm64 and https://git.cloudron.io/cloudron/docker-base-image/-/commits/arm64On top of this, I am trying to implement a better provisioning workflow for development, this is similar to the hotfix, so it is still aimed towards developers porting stuff to arm. More info on this later.
@nebulon Great work!
I got really caught up with work and personal stuff over the last weeks, so I was not able to keep on working.
On most addon repos, I was creating
arm64branches as well, but most of them were just a matter of changing the base image to not have the hash. I was usingcloudron/baseimageand building it on the pi itself before trying to install cloudron, so I had it tagged locally with that name, and others wouldn't download from docker hub, using the local arm64 one instead.I'll search for any changes I have made over here, but they're not a lot.
I'll post a suymmary tonight.
-
@nebulon Great work!
I got really caught up with work and personal stuff over the last weeks, so I was not able to keep on working.
On most addon repos, I was creating
arm64branches as well, but most of them were just a matter of changing the base image to not have the hash. I was usingcloudron/baseimageand building it on the pi itself before trying to install cloudron, so I had it tagged locally with that name, and others wouldn't download from docker hub, using the local arm64 one instead.I'll search for any changes I have made over here, but they're not a lot.
I'll post a suymmary tonight.
@malvim right, it was same route for me then. I have just managed to get all addons and nextcloud to run. The notes here about mongodb were helpful!
Overall I don't think it will be included in the next release though, maybe as something experimental, but while we have a proof-of-concept now, it will still take quite some time to actually make it proper and of course all apps have to be rebuilt...
-
@malvim right, it was same route for me then. I have just managed to get all addons and nextcloud to run. The notes here about mongodb were helpful!
Overall I don't think it will be included in the next release though, maybe as something experimental, but while we have a proof-of-concept now, it will still take quite some time to actually make it proper and of course all apps have to be rebuilt...
-
@nebulon Let me know when you send are finished with your branch (and how we'd build it differently) and I'll make sure my OpenVPN Client is ready!
-
@lonk Can you make a new post for the VPN Client with the current status? I would like to discuss a bit whether it should be part of the Cloudron box code or an app.
-
@lonk I basically got this working already, there are only a few changes required to make the base system work. That is very minor and has more to do with Ubuntu setup rather than our code base.
However the main reason we have not pursued this further is, that in order to support arm (arm64 to be precise), we have to rebuild all addon docker images for a start and patch up the code which creates addon container accordingly with different image tags and even once that is done, we then have to rebuild all app packages for arm as well, which means a lot of testing and potentially fixing apps upstream. This is a lot of work and this has to be done and tested for every app package update of course.
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I do think it is worth it in the long run to support arm, also because VPS provider start adding arm options and at least the raspberrypi 400 showed okish performance while I was doing the proof-of-concept, however we will need a different way to build packages and run the selenium tests for both architectures reliably. This is currently done manually by us due to the lack of such CI/CD pipelines in place.
So all this is certainly doable but unless we see higher demand, it is hard to justify the extra work and for the time being essentially at least double the work per app update.
-
@lonk I basically got this working already, there are only a few changes required to make the base system work. That is very minor and has more to do with Ubuntu setup rather than our code base.
However the main reason we have not pursued this further is, that in order to support arm (arm64 to be precise), we have to rebuild all addon docker images for a start and patch up the code which creates addon container accordingly with different image tags and even once that is done, we then have to rebuild all app packages for arm as well, which means a lot of testing and potentially fixing apps upstream. This is a lot of work and this has to be done and tested for every app package update of course.
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I do think it is worth it in the long run to support arm, also because VPS provider start adding arm options and at least the raspberrypi 400 showed okish performance while I was doing the proof-of-concept, however we will need a different way to build packages and run the selenium tests for both architectures reliably. This is currently done manually by us due to the lack of such CI/CD pipelines in place.
So all this is certainly doable but unless we see higher demand, it is hard to justify the extra work and for the time being essentially at least double the work per app update.
@nebulon Well, that sounds to me like like not much work needs to be done, I'll do some PoC stuff myself and see the difficulty of converting the app. Docker is supposed to support multi arch so it's a matter of getting all the developer's to support multi arch upstream. So, I'll start submitting tickets for those things. Thanks for telling me where you were at, do you have your POC up anywhere yet?
-
@nebulon Well, that sounds to me like like not much work needs to be done, I'll do some PoC stuff myself and see the difficulty of converting the app. Docker is supposed to support multi arch so it's a matter of getting all the developer's to support multi arch upstream. So, I'll start submitting tickets for those things. Thanks for telling me where you were at, do you have your POC up anywhere yet?
-
@lonk I basically got this working already, there are only a few changes required to make the base system work. That is very minor and has more to do with Ubuntu setup rather than our code base.
However the main reason we have not pursued this further is, that in order to support arm (arm64 to be precise), we have to rebuild all addon docker images for a start and patch up the code which creates addon container accordingly with different image tags and even once that is done, we then have to rebuild all app packages for arm as well, which means a lot of testing and potentially fixing apps upstream. This is a lot of work and this has to be done and tested for every app package update of course.
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I do think it is worth it in the long run to support arm, also because VPS provider start adding arm options and at least the raspberrypi 400 showed okish performance while I was doing the proof-of-concept, however we will need a different way to build packages and run the selenium tests for both architectures reliably. This is currently done manually by us due to the lack of such CI/CD pipelines in place.
So all this is certainly doable but unless we see higher demand, it is hard to justify the extra work and for the time being essentially at least double the work per app update.
@nebulon said in Cloudron on a Raspberry pi?:
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I have some experience with this and have set up my own multi-arch go build pipelines using a single Dockerfile for some of my other apps: minitor-go, dockron, tag-checker, and for Python ones too: original minitor.
Here's a sample repo demonstrating my process: multiarch-pipeline-test. It's easier these days if your server has
docker buildxthough.Also, since with Cloudron we're most often building things that exist upstream, here's an example multi-arch build repo I have for the Golang project cadvisor. It will auto build a particular cadvisor version on a git tag so I just need to create a release on my Gitea server and the build is started and deployed. With cadvisor, I have to clone the whole repo and cross-compile the cadvisor binary for arm becaue there is no pre-compiled binary. If there is, it should be even easier to just pull that binary.
Anyway, I'm happy to help if there are any applications that may be critical to be ported.
-
@nebulon said in Cloudron on a Raspberry pi?:
To give one simple example, any app using the go language, where we take the release builds, has to get some logic or separate Dockerfile to deal with arm.
I have some experience with this and have set up my own multi-arch go build pipelines using a single Dockerfile for some of my other apps: minitor-go, dockron, tag-checker, and for Python ones too: original minitor.
Here's a sample repo demonstrating my process: multiarch-pipeline-test. It's easier these days if your server has
docker buildxthough.Also, since with Cloudron we're most often building things that exist upstream, here's an example multi-arch build repo I have for the Golang project cadvisor. It will auto build a particular cadvisor version on a git tag so I just need to create a release on my Gitea server and the build is started and deployed. With cadvisor, I have to clone the whole repo and cross-compile the cadvisor binary for arm becaue there is no pre-compiled binary. If there is, it should be even easier to just pull that binary.
Anyway, I'm happy to help if there are any applications that may be critical to be ported.
@iamthefij nice thanks for the pointers, when we got started on this this will help. I tried to use
docker buildxbut I couldn't get it to work and produce binaries which would actually run on the raspberrypi, so I ended up using the raspberrypi to build the images itself, which surprisingly showed how beefy that board has become -
@iamthefij nice thanks for the pointers, when we got started on this this will help. I tried to use
docker buildxbut I couldn't get it to work and produce binaries which would actually run on the raspberrypi, so I ended up using the raspberrypi to build the images itself, which surprisingly showed how beefy that board has become -
@nebulon careful with compiling on rPi's as they can overheat and burn out
Having a small heatsink or fan handy helps a lot. -
@iamthefij nice thanks for the pointers, when we got started on this this will help. I tried to use
docker buildxbut I couldn't get it to work and produce binaries which would actually run on the raspberrypi, so I ended up using the raspberrypi to build the images itself, which surprisingly showed how beefy that board has become -
@iamthefij nice thanks for the pointers, when we got started on this this will help. I tried to use
docker buildxbut I couldn't get it to work and produce binaries which would actually run on the raspberrypi, so I ended up using the raspberrypi to build the images itself, which surprisingly showed how beefy that board has become@nebulon would all binaries not run (eg.
/bin/shfrom within the base) or just Go binaries that you compiled within thebuildxpipeline? If it's the former, it may not be using the right base image. If it's the latter and the former works, perhaps setting theGOARCHvariable via a build-arg would solve it.Note: I personally have not used
buildxyet, but from what I can see it's a simpler, automatic version of what I'm trying to do with qemu that handles the manifest for you. So I think you should just be able to build without the muckiness of all the build-args I pass, but if not you can play with mixing those in until it works.I think
buildxis supported on my laptop, so I can give it a try, but it's not supported on my CI box yet, so I haven't switched. -
@nebulon would all binaries not run (eg.
/bin/shfrom within the base) or just Go binaries that you compiled within thebuildxpipeline? If it's the former, it may not be using the right base image. If it's the latter and the former works, perhaps setting theGOARCHvariable via a build-arg would solve it.Note: I personally have not used
buildxyet, but from what I can see it's a simpler, automatic version of what I'm trying to do with qemu that handles the manifest for you. So I think you should just be able to build without the muckiness of all the build-args I pass, but if not you can play with mixing those in until it works.I think
buildxis supported on my laptop, so I can give it a try, but it's not supported on my CI box yet, so I haven't switched.
So far, there's not much interest in ARM server (apart from hosting on pi), I was hoping we will see more ARM servers mainstream...