How does Cloudron work? What does it do? etc :)

jdaviescoates

I'm working with @thetomester13 on setting up a hosted Cloudron service (see https://selfhost.cloud ) and we'd like to explain a bit about how Cloudron works (and why the way it works is so cool/ great/ good for security etc).

I think it'd be really great/ useful to have such info on the Cloudron website too! (I guess as a start perhaps copy/ paste some of what's in the box README https://git.cloudron.io/cloudron/box/-/blob/master/README.md )

My very limited understanding goes something like this:

Cloudron is largely based on Docker. This is good because it's more secure and if one app goes down/ gets hacked it can't effect all the other apps on the server.

(I'd love to be able to fill this bit out a bit with stuff about why Docker/ containerisation is cool/ good/ secure - I guess maybe from https://www.docker.com/why-docker and https://www.docker.com/resources/what-container etc )

But Cloudron is not just Docker.

Cloudron packages apps into Docker containers, but they also include some other magic/ goodness, including tests. What else?

I think some of the what else are relevant Cloudron addons and other goodness like the STUN/ TURN server, LDAP server, Email server, Firewall, DNS stuff etc.

Which are the main/ most important ones? and what does each one do?

And then all this magic means it's really super easy to selfhost awesome apps on your own domains and to have LDAP powered SSO etc, plus to keep them all updated, backed-up and secure.

Right?

Basically how does Cloudron work? What does the box code do? why is that awesome?

Thanks!

vjvanjungg

@jdaviescoates for the very layman non-dev users, i would describe Cloudron simply to them as an App Store like CHPlay or Apple AppStore. With an email or emails of your own domains. The best thing is they keep the bad guys away (an important thing for young, new publishers). I get satisfaction when I see messages like ‘Connection from ... denied’ in the email logs cause I feel safe that these guys are protecting me. Sure this doesn’t touch 10% what Cloudron power does but gradually they’ll see.

robi

Check out the talk @girish did at my Meetup on this very topic.

Self-Hosting For Mortals - Effortlessly run Apps on your Servers using Cloudron
https://www.meetup.com/BayLISA/events/pljmfryzlbtb/

jdaviescoates

@robi thanks, sounds perfect!

This month at BayLISA we will learn about Cloudron from Girish Ramakrishnan.

Girish is the co-founder of Cloudron. He is a privacy enthusiast and has this crazy idea that in an enlightened future, everyone will own and operate a smartserver to accompany their smartphone. This will give them complete control and ownership of their data. He is an engineer by trade and has contributed to many FOSS projects including Qt, WebKit, KDE, Crosswalk and Haraka. These days he copy/pastes code from the internet and calls himself a full stack developer.

About the Talk:
Today, all of us have a smartphone in our pocket. These supercomputers have more computing power than the ones used by NASA during the Apollo mission. Everyone and their mom can install apps on their phones. Apps are deployed sandboxed and kept automatically updated. Nobody thinks about "managing" a phone.

The same experience does not exist for the server market. While servers can be spun up within seconds at any public cloud provider, installing apps like a blog, a chat, a wiki remains a time consuming and daunting task. The constant effort required to keep apps up-to-date and secure deters most people from self-hosting.

In this talk, we will cover:

Why self-host at all?

The challenges to self-hosting apps

How containers provides a unique app delivery and deployment mechanism

How the Cloudron solution greatly simplifies managing apps on your server. This will be very technical and we will cover the Cloudron architecture in detail.

But I don't see anywhere I can actually watch the talk! Can I watch it somewhere?

jdaviescoates

@jdaviescoates said in How does Cloudron work? What does it do? etc :

But I don't see anywhere I can actually watch the talk! Can I watch it somewhere?

Ah, I had found the https://www.pscp.tv/w/1YqKDBZVqjvJV link on that meetup page, but in Firefox nothing seems to happen when I visit it, so I assumed it was just a live thing with no ability to watch back... but then I tried it in Chromium and it worked, thanks! Will watch...

robi

@jdaviescoates glad you figured it out!

jdaviescoates

@jdaviescoates said in How does Cloudron work? What does it do? etc :

Ah, I had found the https://www.pscp.tv/w/1YqKDBZVqjvJV link on that meetup page, but in Firefox nothing seems to happen when I visit it, so I assumed it was just a live thing with no ability to watch back... but then I tried it in Chromium and it worked, thanks! Will watch...

@Staff you should add this video to your peertube.

https://pscp.download/?url=https%3A%2F%2Fwww.pscp.tv%2Fw%2F1YqKDBZVqjvJV

jdaviescoates

I just watch the video linked above (after using OpenShot video to rotate it 90 degrees to the left to avoid getting a crook neck!) here are my notes:

Benefits of Cloudron

Quick and Easy to install and configure properly apps on your own domains
Easy to update
Easy to maintain
Secure by default
Control of your data - you know and decide where it is stored, and who and how the data can be accessed

Selfhosting = you deploy apps on servers of your choosing

DNS management and SSL certificates all sorted using LetsEncrypt

Convenience of SaaS with the control of a private cloud

with a user experience similar to what people are now used to with their smart phones - everyone can easyily install apps from the app store and apps are automatically updated - we don't have to maintain our phones, it just works.

Smart phone -> Smart server

Full Email server built in!!!

Can easily move (or clone) apps from one domain to another

== Architecture: ==

Cloudron is install on the server
Apps are listed from Cloudron's App Store
Cloudron install the App (it's not something the App Store does)
Cloudron periodically checks for updates
Each Cloudron installation is independent and private. Cloudron don't have access to the server. They just provie the App Store list and provide updates for apps

== Anatomy of an app ==

2 parts:

Docker based packaging

dependency management (differnt apps require different versions of PHP, Ruby, whatever Docker lets you package them all together in a single package)
Static configuration (decision about whether to use nginx or apache etc are taken care of)
Density (used to have to use one virtual machine for each app, with Docker you can have loads of differnt apps on one server in containers)

Mainfest file (information about the things the app needs to run)

Addons: which database to use (MySql, Postgres, etc), Auth, Email
Port bindings
Version: Title, Icon, Description, Author

== App Store ==

Just like Google Play story, Appple app store

its just a Distribution mechanism, doesn't have access to the server.
holds manifest meta data (what apps needs to run)
versioning information

All the app packages are open source, so you

(not relevant for selfhost.cloud but it handles Dynamic DNS for people hosting at home)

== The Platform (the bit that is running on your server) ==

Very similar to Heroku - you can give it code and it'll run it for you

Each addon is a micro-service (and can say, this app needs mysql server, this app needs to be able to send email etc)
Addon access credentials as env var (environment variables)

(ask Girish for the slides)

All the addons (databases, email etc) are running in their own Docker containers and operate like mirco-services, you can ask them to create/ destroy databases etc.

== App Lifecycle ==

Install
-- Configure DNS
-- Downloads docker image and manifest file from the Cloudron app store
-- Sets up addons
-- Logrotate, Collect (stats about the app, how much CPU, memory it using etc), Firewall
-- Runs the docker container
--- Dynamic configuration (giving the app db credentials, SMTP credentials to send email, e.g. for WordPress it creates the wp-config.php file with all the relevant credentials)
-- Gets SSL certificates from LetsEncrypt (and set-up reverse proxy, i.e. when blog.domain.com is visited forward to this container)
Updates
-- Read only and stateless app containers (all apps are read only - apps cannot write to their file system, if they could then users could add all sorts of random files and it wouldn't be possible to update smoothly, the code cannot be modified. This means when there is an update we can just throw away the old container and get in the new container. So where does the app write stuff it needs to? We let the app write in 3 locations 1) /tmp for temporary files 2) /run which contains runtime files which an app needs to communication across various processes, 3) app/data/ where the app will put images, files uploaded etc - everything in app data is part of the backup, /tmp and /run is not backed up)
-- Rolling updates
-- Signed releases
-- Selenium based tests (test that everything actually still works)

== Maintainence ==

Backups
-- Per app backups (means you can roll back just that app instead of all the apps on the server, like would happen if you just rolled back to the server snapshot).
-- Backup only addon data (don't need to back up the docker container because it's read only, nothing has change, only need to back up databases etc, plus the /app/data directory)
-- Apps can be trivially cloned and rolled back
-- Can be stored offsite to S3, DO Spaces, etc
Alerts
-- Email notifications

== Security ==

Turn-key security
HTTPS only
SSL, HSTS
App isolation and sandboxed (apps can only access their databases etc, not the other databases etc)
Rate limits, Activity logs (built in standard security practices e.g. Rate limtes: if people try to login too quickly, login from a new device etc, Activity logs: everything that happening on the system, who is doing what)
Signed releases
More info at https://docs.cloudron.io/security/

Who are Cloudron customers:

20-30% individuals
20-30% tech startups
universities (but they have differnt pricing structures)
very popular with ngos and cooperatives, in Europe (France, Germany) - ha! Girish described co-ops as new! hehe
also resellers and hosting providers

@thetomester13 a slight red flag re our Selfhost.Cloud plans - Cloudron started off providing managed Cloudron, but users wanted to install it on their own servers - still, I think they are likely still plenty of people who would like it as a service too. Just like people still sometimes pay me to create selfhosted WordPress sites for them and still use WordPress.com even those self-hosting WordPress is really easy.

@girish do you still have the slides you used in the talk? could you perhaps share them here? (even though I've more or less transcribed them plus added a load of stuff you said that isn't on the slides) Thanks!

girish

@jdaviescoates I will probably upload it in some place with a memorable URL later but these are the slides - https://files.cloudron.io/s/y6NinTqKdFHaBGi . Thanks for the elaborate notes!

marcusquinn

Adding a note the convo without completely reading but if there's effort to build a video library to promote and educate on features, I'd suggest aiming to segment into 2-5min videos per topic, attention-spans online aren't long enough for 30 min tutorials for the average prospect.

jdaviescoates

@marcusquinn and to add to that, any video that's going to be shared on socials (where most people discover video) really needs to be very clear what it's about and have a hook in the first 10 seconds (most videos aren't watch past that - if a video doesn't grab you in the first 10 seconds, you don't watch it).

jdaviescoates

@girish thanks!

plusone-nick

@jdaviescoates I have been messing around with a few "intro to cloudron" scripts, testing out OBS backdrops and mic settings. Will share a draft when I have a legit one compiled =] Hopefully sooner than later lol

jdaviescoates

@plusone-nick nice, looking forward to seeing it, thanks

nebulon

@jdaviescoates I rotated the video now and uploaded it to https://videos.cloudron.io/videos/watch/79a34d05-a60b-4ec3-9327-fd736016494c

robi

@jdaviescoates nicely done.

What is the file size?

Can you share with me the edited video, so I can post it to our new BayLISA.org YT channel?

Perhaps over a FilePizza link or whatever works for you.

robi

@nebulon ooh, thank you, just saw this! I'll see if we can use this one.

jdaviescoates

@robi yeah I think @nebulon has done a better job of rotating than I did. I guess he probably just used some command line tools? My file ended up over 3gb and had black space all around the edges.

nebulon

Indeed fyi I used:

ffmpeg -i BayLISA-Cloudron.mp4 -vf "transpose=2" BayLISA-Cloudron-rotated.mp4

the value 2 here is for 90° counter clock-wise, it took me 2 attempts to get the right value...happens when you don't read the docs

robi

@nebulon How big was the file?

nebulon

@robi 88MB looks like, the original was 180MB so I guess it also did some compression alongside. Overall the quality is not really stellar, but luckily the sound is fine.

robi

@nebulon great, check you email please

jdaviescoates

@staff is there a handy list of all the addons Cloudron currently has? Ideally with a brief description of what they do/ what they are for? Thanks

mehdi

@jdaviescoates https://docs.cloudron.io/custom-apps/addons/

jdaviescoates

@mehdi doh, thanks!

jdaviescoates

@nebulon said in How does Cloudron work? What does it do? etc :

@jdaviescoates I rotated the video now and uploaded it to https://videos.cloudron.io/videos/watch/79a34d05-a60b-4ec3-9327-fd736016494c

Just realised you're missing about half an hour on that video!

robi

@jdaviescoates Nice catch! You're right, seems to be cut off at the end.

nebulon

@robi indeed, have to see why that happened So the root cause is that either the video on persicope is cut off or the download from periscope just stops at some point. Not sure, however I wasn't able to fully download it thus far.

jdaviescoates

@nebulon I've got the complete original from periscope if that helps?

When I tried Alltube, it didn't work, but that periscope download link I shared above worked for me.

This one https://pscp.download/?url=https%3A%2F%2Fwww.pscp.tv%2Fw%2F1YqKDBZVqjvJV

girish

@jdaviescoates how long is the video length. The one I downloaded (a week ago) directly is 31:26

jdaviescoates

@girish that's even shorter than the one @nebulon uploaded! The full thing is about 1hr 15 minute

girish

@jdaviescoates Do you think you can upload the have somewhere, so we can download it? I tried periscope again and only got a 30min video.

robi

@jdaviescoates I'd suggest a file.pizza link so we can help seed/speed it around.

jdaviescoates

@girish yes will do once back on my laptop where the file is...

jdaviescoates

@girish @robi there is clearly something very wrong with the timestamps in the original hence why we've got all these different timing!

Anyways, here is the 295.2MB original I managed to get off Periscope (which says it's 1:16:26 when I open it with VLC):

(well, this is taking forever to process for some reason, so will come back and add the link if it ever finishes)

FilePizza still says processing so here it is via Surfer:
https://files.uniteddiversity.coop/BayLISA_Cloudron.mp4 (although again, something strange about this video as Surfer wont play it when it normally does play videos - plays fine for me in VLC though)

And here is a 147.4MB rotated version (using @nebulon's command) which is bizarrely 1:19:24

https://transfer.uniteddiversity.coop/beef/spinach/oregano/broccoli

girish

@jdaviescoates thanks, I managed to download the unrotated version. I will try to rotate and upload it now.

girish

@jdaviescoates Thanks, file.pizza delivered! I have re-uploaded to https://videos.cloudron.io/videos/watch/206fc4b7-9641-4453-8be0-67450f2574b1

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

How does Cloudron work? What does it do? etc :)