How does Cloudron work? What does it do? etc :)
-
I'm working with @thetomester13 on setting up a hosted Cloudron service (see https://selfhost.cloud ) and we'd like to explain a bit about how Cloudron works (and why the way it works is so cool/ great/ good for security etc).
I think it'd be really great/ useful to have such info on the Cloudron website too! (I guess as a start perhaps copy/ paste some of what's in the box README https://git.cloudron.io/cloudron/box/-/blob/master/README.md )
My very limited understanding goes something like this:
Cloudron is largely based on Docker. This is good because it's more secure and if one app goes down/ gets hacked it can't effect all the other apps on the server.
(I'd love to be able to fill this bit out a bit with stuff about why Docker/ containerisation is cool/ good/ secure - I guess maybe from https://www.docker.com/why-docker and https://www.docker.com/resources/what-container etc )
But Cloudron is not just Docker.
Cloudron packages apps into Docker containers, but they also include some other magic/ goodness, including tests. What else?
I think some of the what else are relevant Cloudron addons and other goodness like the STUN/ TURN server, LDAP server, Email server, Firewall, DNS stuff etc.
Which are the main/ most important ones? and what does each one do?
And then all this magic means it's really super easy to selfhost awesome apps on your own domains and to have LDAP powered SSO etc, plus to keep them all updated, backed-up and secure.
Right?
Basically how does Cloudron work? What does the box code do? why is that awesome?
Thanks!
-
@jdaviescoates for the very layman non-dev users, i would describe Cloudron simply to them as an App Store like CHPlay or Apple AppStore. With an email or emails of your own domains. The best thing is they keep the bad guys away (an important thing for young, new publishers). I get satisfaction when I see messages like ‘Connection from ... denied’ in the email logs cause I feel safe that these guys are protecting me. Sure this doesn’t touch 10% what Cloudron power does but gradually they’ll see.
-
Check out the talk @girish did at my Meetup on this very topic.
- Self-Hosting For Mortals - Effortlessly run Apps on your Servers using Cloudron
https://www.meetup.com/BayLISA/events/pljmfryzlbtb/
- Self-Hosting For Mortals - Effortlessly run Apps on your Servers using Cloudron
-
@robi thanks, sounds perfect!
This month at BayLISA we will learn about Cloudron from Girish Ramakrishnan.
Girish is the co-founder of Cloudron. He is a privacy enthusiast and has this crazy idea that in an enlightened future, everyone will own and operate a smartserver to accompany their smartphone. This will give them complete control and ownership of their data. He is an engineer by trade and has contributed to many FOSS projects including Qt, WebKit, KDE, Crosswalk and Haraka. These days he copy/pastes code from the internet and calls himself a full stack developer.
About the Talk:
Today, all of us have a smartphone in our pocket. These supercomputers have more computing power than the ones used by NASA during the Apollo mission. Everyone and their mom can install apps on their phones. Apps are deployed sandboxed and kept automatically updated. Nobody thinks about "managing" a phone.The same experience does not exist for the server market. While servers can be spun up within seconds at any public cloud provider, installing apps like a blog, a chat, a wiki remains a time consuming and daunting task. The constant effort required to keep apps up-to-date and secure deters most people from self-hosting.
In this talk, we will cover:
- Why self-host at all?
- The challenges to self-hosting apps
- How containers provides a unique app delivery and deployment mechanism
- How the Cloudron solution greatly simplifies managing apps on your server. This will be very technical and we will cover the Cloudron architecture in detail.
But I don't see anywhere I can actually watch the talk! Can I watch it somewhere?
-
@jdaviescoates said in How does Cloudron work? What does it do? etc
:But I don't see anywhere I can actually watch the talk! Can I watch it somewhere?
Ah, I had found the https://www.pscp.tv/w/1YqKDBZVqjvJV link on that meetup page, but in Firefox nothing seems to happen when I visit it, so I assumed it was just a live thing with no ability to watch back... but then I tried it in Chromium and it worked, thanks! Will watch...
-
@jdaviescoates glad you figured it out!
-
@jdaviescoates said in How does Cloudron work? What does it do? etc
:Ah, I had found the https://www.pscp.tv/w/1YqKDBZVqjvJV link on that meetup page, but in Firefox nothing seems to happen when I visit it, so I assumed it was just a live thing with no ability to watch back... but then I tried it in Chromium and it worked, thanks! Will watch...
@Staff you should add this video to your peertube.
https://pscp.download/?url=https%3A%2F%2Fwww.pscp.tv%2Fw%2F1YqKDBZVqjvJV
-
I just watch the video linked above (after using OpenShot video to rotate it 90 degrees to the left to avoid getting a crook neck!) here are my notes:
Benefits of Cloudron
Quick and Easy to install and configure properly apps on your own domains
Easy to update
Easy to maintain
Secure by default
Control of your data - you know and decide where it is stored, and who and how the data can be accessedSelfhosting = you deploy apps on servers of your choosing
DNS management and SSL certificates all sorted using LetsEncrypt
Convenience of SaaS with the control of a private cloud
- with a user experience similar to what people are now used to with their smart phones - everyone can easyily install apps from the app store and apps are automatically updated - we don't have to maintain our phones, it just works.
Smart phone -> Smart server
Full Email server built in!!!
Can easily move (or clone) apps from one domain to another
== Architecture: ==
Cloudron is install on the server
Apps are listed from Cloudron's App Store
Cloudron install the App (it's not something the App Store does)
Cloudron periodically checks for updates
Each Cloudron installation is independent and private. Cloudron don't have access to the server. They just provie the App Store list and provide updates for apps== Anatomy of an app ==
2 parts:
- Docker based packaging
- dependency management (differnt apps require different versions of PHP, Ruby, whatever Docker lets you package them all together in a single package)
- Static configuration (decision about whether to use nginx or apache etc are taken care of)
- Density (used to have to use one virtual machine for each app, with Docker you can have loads of differnt apps on one server in containers)
- Mainfest file (information about the things the app needs to run)
- Addons: which database to use (MySql, Postgres, etc), Auth, Email
- Port bindings
- Version: Title, Icon, Description, Author
== App Store ==
Just like Google Play story, Appple app store
- its just a Distribution mechanism, doesn't have access to the server.
- holds manifest meta data (what apps needs to run)
- versioning information
All the app packages are open source, so you
(not relevant for selfhost.cloud but it handles Dynamic DNS for people hosting at home)
== The Platform (the bit that is running on your server) ==
Very similar to Heroku - you can give it code and it'll run it for you
- Each addon is a micro-service (and can say, this app needs mysql server, this app needs to be able to send email etc)
- Addon access credentials as env var (environment variables)
(ask Girish for the slides)
All the addons (databases, email etc) are running in their own Docker containers and operate like mirco-services, you can ask them to create/ destroy databases etc.
== App Lifecycle ==
-
Install
-- Configure DNS
-- Downloads docker image and manifest file from the Cloudron app store
-- Sets up addons
-- Logrotate, Collect (stats about the app, how much CPU, memory it using etc), Firewall
-- Runs the docker container
--- Dynamic configuration (giving the app db credentials, SMTP credentials to send email, e.g. for WordPress it creates the wp-config.php file with all the relevant credentials)
-- Gets SSL certificates from LetsEncrypt (and set-up reverse proxy, i.e. when blog.domain.com is visited forward to this container) -
Updates
-- Read only and stateless app containers (all apps are read only - apps cannot write to their file system, if they could then users could add all sorts of random files and it wouldn't be possible to update smoothly, the code cannot be modified. This means when there is an update we can just throw away the old container and get in the new container. So where does the app write stuff it needs to? We let the app write in 3 locations 1) /tmp for temporary files 2) /run which contains runtime files which an app needs to communication across various processes, 3) app/data/ where the app will put images, files uploaded etc - everything in app data is part of the backup, /tmp and /run is not backed up)
-- Rolling updates
-- Signed releases
-- Selenium based tests (test that everything actually still works)
== Maintainence ==
-
Backups
-- Per app backups (means you can roll back just that app instead of all the apps on the server, like would happen if you just rolled back to the server snapshot).
-- Backup only addon data (don't need to back up the docker container because it's read only, nothing has change, only need to back up databases etc, plus the /app/data directory)
-- Apps can be trivially cloned and rolled back
-- Can be stored offsite to S3, DO Spaces, etc -
Alerts
-- Email notifications
== Security ==
- Turn-key security
- HTTPS only
- SSL, HSTS
- App isolation and sandboxed (apps can only access their databases etc, not the other databases etc)
- Rate limits, Activity logs (built in standard security practices e.g. Rate limtes: if people try to login too quickly, login from a new device etc, Activity logs: everything that happening on the system, who is doing what)
- Signed releases
- More info at https://docs.cloudron.io/security/
Who are Cloudron customers:
20-30% individuals
20-30% tech startups
universities (but they have differnt pricing structures)
very popular with ngos and cooperatives, in Europe (France, Germany) - ha! Girish described co-ops as new! hehe
also resellers and hosting providers@thetomester13 a slight red flag re our Selfhost.Cloud plans - Cloudron started off providing managed Cloudron, but users wanted to install it on their own servers - still, I think they are likely still plenty of people who would like it as a service too. Just like people still sometimes pay me to create selfhosted WordPress sites for them and still use WordPress.com even those self-hosting WordPress is really easy.
@girish do you still have the slides you used in the talk? could you perhaps share them here? (even though I've more or less transcribed them plus added a load of stuff you said that isn't on the slides) Thanks!
-
@jdaviescoates I will probably upload it in some place with a memorable URL later but these are the slides - https://files.cloudron.io/s/y6NinTqKdFHaBGi . Thanks for the elaborate notes!
-
Adding a note the convo without completely reading but if there's effort to build a video library to promote and educate on features, I'd suggest aiming to segment into 2-5min videos per topic, attention-spans online aren't long enough for 30 min tutorials for the average prospect.
We're not here for a long time - but we are here for a good time :)
Jersey/UK
Work & Ecommerce Advice: https://brandlight.org
Personal & Software Tips: https://marcusquinn.com -
@marcusquinn and to add to that, any video that's going to be shared on socials (where most people discover video) really needs to be very clear what it's about and have a hook in the first 10 seconds (most videos aren't watch past that - if a video doesn't grab you in the first 10 seconds, you don't watch it).
-
@girish thanks!
-
@jdaviescoates I have been messing around with a few "intro to cloudron" scripts, testing out OBS backdrops and mic settings. Will share a draft when I have a legit one compiled =] Hopefully sooner than later lol
✌💙+1
-
@plusone-nick nice, looking forward to seeing it, thanks
-
@jdaviescoates I rotated the video now and uploaded it to https://videos.cloudron.io/videos/watch/79a34d05-a60b-4ec3-9327-fd736016494c
-
@jdaviescoates nicely done.
What is the file size?
Can you share with me the edited video, so I can post it to our new BayLISA.org YT channel?
Perhaps over a FilePizza link or whatever works for you.
-
@nebulon ooh, thank you, just saw this! I'll see if we can use this one.
-
-
Indeed fyi I used:
ffmpeg -i BayLISA-Cloudron.mp4 -vf "transpose=2" BayLISA-Cloudron-rotated.mp4the value 2 here is for 90° counter clock-wise, it took me 2 attempts to get the right value...happens when you don't read the docs
-
@nebulon How big was the file?
-
@robi 88MB looks like, the original was 180MB so I guess it also did some compression alongside. Overall the quality is not really stellar, but luckily the sound is fine.
-
@nebulon great, check you email please
-
@staff is there a handy list of all the addons Cloudron currently has? Ideally with a brief description of what they do/ what they are for? Thanks
-
-
@mehdi doh, thanks!
-
@nebulon said in How does Cloudron work? What does it do? etc
:@jdaviescoates I rotated the video now and uploaded it to https://videos.cloudron.io/videos/watch/79a34d05-a60b-4ec3-9327-fd736016494c
Just realised you're missing about half an hour on that video!
-
@jdaviescoates Nice catch! You're right, seems to be cut off at the end.
-
@robi indeed, have to see why that happened
So the root cause is that either the video on persicope is cut off or the download from periscope just stops at some point. Not sure, however I wasn't able to fully download it thus far. -
@nebulon I've got the complete original from periscope if that helps?
When I tried Alltube, it didn't work, but that periscope download link I shared above worked for me.
This one https://pscp.download/?url=https%3A%2F%2Fwww.pscp.tv%2Fw%2F1YqKDBZVqjvJV
-
@jdaviescoates how long is the video length. The one I downloaded (a week ago) directly is 31:26
-
-
@jdaviescoates Do you think you can upload the have somewhere, so we can download it? I tried periscope again and only got a 30min video.
-
@jdaviescoates I'd suggest a file.pizza link so we can help seed/speed it around.
-
@girish yes will do once back on my laptop where the file is...
-
@girish @robi there is clearly something very wrong with the timestamps in the original hence why we've got all these different timing!
Anyways, here is the 295.2MB original I managed to get off Periscope (which says it's 1:16:26 when I open it with VLC):
(well, this is taking forever to process for some reason, so will come back and add the link if it ever finishes)
FilePizza still says processing so here it is via Surfer:
https://files.uniteddiversity.coop/BayLISA_Cloudron.mp4 (although again, something strange about this video as Surfer wont play it when it normally does play videos - plays fine for me in VLC though)And here is a 147.4MB rotated version (using @nebulon's command) which is bizarrely 1:19:24
https://transfer.uniteddiversity.coop/beef/spinach/oregano/broccoli
-
@jdaviescoates thanks, I managed to download the unrotated version. I will try to rotate and upload it now.
-
@jdaviescoates Thanks, file.pizza delivered! I have re-uploaded to https://videos.cloudron.io/videos/watch/206fc4b7-9641-4453-8be0-67450f2574b1
-
@girish I have a small question about this video I've been meaning to ask for ages... somewhere you mention that Cloudron uses "Linux containers" - does that basically just mean Docker? or does Cloudron also use some other form of Linux containerization too? thanks!
-
@jdaviescoates all apps and services use docker for containerization currently. When we started, we had ideas of allow this to maybe change that, as Cloudron acts as an abstraction on top of such lower-level features. In the end so far there was no real reason for supporting other containerization methods. Also it would cause mostly unnecessary extra work for little benefit, still we try to not expose containerization specifics to apps to stay portable as much as possible for the future.
-
@jdaviescoates The linux kernel has various built-in features/system calls like 'cgroups' (provides cpu/ram/resource isolation), namespaces (pid/ipc/mounts), overlay filesystem etc. These things apply to 'process'es.
'Container' is a term and not some feature of the kernel as such. It brings all the above system calls together along with bundling/packaging aspect to help create containerized "apps". Apps can be desktop UI, web apps, daemons, anything.
Docker is one implementation of the 'container' concept. When creating a implementation (just like how you design a language), you have to decide style/structure/format. We have this Dockerfile that explains how to put deps together, how to configure things, how to push the final build image to the cloud and pull down this image and deploy it on another server etc.
Other implementations exist which are in various states - rocket, OpenVZ(?), LXC, snap, containerd, cri-o off my head.
When we started, Docker was the de-facto implementation. These days there are more options and in theory we can switch to another one without affecting end users (though it's a lot of work). Also, when we started, we had no 'custom apps'. Changing the run time will break custom apps, so I think we are married to Docker until that project lives.
-
Forgot to mention sandstom.io, which implements containers by itself and does not use docker.
-
@girish said in How does Cloudron work? What does it do? etc
:Changing the run time will break custom apps, so I think we are married to Docker until that project lives.
That's not quite true.. remember Nestybox?
Their runtime called "sysbox" is a drop in replacement forruncwhich docker uses by default. -
@robi From what I remember, sysbox was like a backend to docker, it doesn't replace docker. Unlike the other things I mentioned which will replace docker.
-
@girish you should take a look at it again.
-
@robi Would be interesting to know what
systemd-detect-virtprints for nestybox -
I just found that https://www.freedesktop.org/software/systemd/man/systemd-detect-virt.html has quite a nice list of container and vm technologies.
-
@girish said in How does Cloudron work? What does it do? etc
:@robi Would be interesting to know what
systemd-detect-virtprints for nestyboxThat's easy to test, just change one line in the docker config or specify it on the CLI and bring up a container using sysbox, then run the tool.
