How does Cloudron work? What does it do? etc :)

[jdaviescoates]

I'm working with @thetomester13 on setting up a hosted Cloudron service (see https://selfhost.cloud ) and we'd like to explain a bit about how Cloudron works (and why the way it works is so cool/ great/ good for security etc).

I think it'd be really great/ useful to have such info on the Cloudron website too! (I guess as a start perhaps copy/ paste some of what's in the box README https://git.cloudron.io/cloudron/box/-/blob/master/README.md )

My very limited understanding goes something like this:

Cloudron is largely based on Docker. This is good because it's more secure and if one app goes down/ gets hacked it can't effect all the other apps on the server.

(I'd love to be able to fill this bit out a bit with stuff about why Docker/ containerisation is cool/ good/ secure - I guess maybe from https://www.docker.com/why-docker and https://www.docker.com/resources/what-container etc )

But Cloudron is not just Docker.

Cloudron packages apps into Docker containers, but they also include some other magic/ goodness, including tests. What else?

I think some of the what else are relevant Cloudron addons and other goodness like the STUN/ TURN server, LDAP server, Email server, Firewall, DNS stuff etc.

Which are the main/ most important ones? and what does each one do?

And then all this magic means it's really super easy to selfhost awesome apps on your own domains and to have LDAP powered SSO etc, plus to keep them all updated, backed-up and secure.

Right?

Basically how does Cloudron work? What does the box code do? why is that awesome?

Thanks!

[vjvanjungg]

@jdaviescoates for the very layman non-dev users, i would describe Cloudron simply to them as an App Store like CHPlay or Apple AppStore. With an email or emails of your own domains. The best thing is they keep the bad guys away (an important thing for young, new publishers). I get satisfaction when I see messages like ‘Connection from ... denied’ in the email logs cause I feel safe that these guys are protecting me. Sure this doesn’t touch 10% what Cloudron power does but gradually they’ll see.

[robi]

Check out the talk @girish did at my Meetup on this very topic.

• Self-Hosting For Mortals - Effortlessly run Apps on your Servers using Cloudron
  https://www.meetup.com/BayLISA/events/pljmfryzlbtb/

[jdaviescoates]

@robi thanks, sounds perfect!

This month at BayLISA we will learn about Cloudron from Girish Ramakrishnan.

Girish is the co-founder of Cloudron. He is a privacy enthusiast and has this crazy idea that in an enlightened future, everyone will own and operate a smartserver to accompany their smartphone. This will give them complete control and ownership of their data. He is an engineer by trade and has contributed to many FOSS projects including Qt, WebKit, KDE, Crosswalk and Haraka. These days he copy/pastes code from the internet and calls himself a full stack developer.

About the Talk:
Today, all of us have a smartphone in our pocket. These supercomputers have more computing power than the ones used by NASA during the Apollo mission. Everyone and their mom can install apps on their phones. Apps are deployed sandboxed and kept automatically updated. Nobody thinks about "managing" a phone.

The same experience does not exist for the server market. While servers can be spun up within seconds at any public cloud provider, installing apps like a blog, a chat, a wiki remains a time consuming and daunting task. The constant effort required to keep apps up-to-date and secure deters most people from self-hosting.

In this talk, we will cover:

• Why self-host at all?
• The challenges to self-hosting apps
• How containers provides a unique app delivery and deployment mechanism
• How the Cloudron solution greatly simplifies managing apps on your server. This will be very technical and we will cover the Cloudron architecture in detail.

But I don't see anywhere I can actually watch the talk! Can I watch it somewhere?

[jdaviescoates]

@jdaviescoates said in How does Cloudron work? What does it do? etc :

But I don't see anywhere I can actually watch the talk! Can I watch it somewhere?

Ah, I had found the https://www.pscp.tv/w/1YqKDBZVqjvJV link on that meetup page, but in Firefox nothing seems to happen when I visit it, so I assumed it was just a live thing with no ability to watch back... but then I tried it in Chromium and it worked, thanks! Will watch...

[robi]

@jdaviescoates glad you figured it out!

[jdaviescoates]

@jdaviescoates said in How does Cloudron work? What does it do? etc :

Ah, I had found the https://www.pscp.tv/w/1YqKDBZVqjvJV link on that meetup page, but in Firefox nothing seems to happen when I visit it, so I assumed it was just a live thing with no ability to watch back... but then I tried it in Chromium and it worked, thanks! Will watch...

@Staff you should add this video to your peertube.

https://pscp.download/?url=https%3A%2F%2Fwww.pscp.tv%2Fw%2F1YqKDBZVqjvJV

[jdaviescoates]

I just watch the video linked above (after using OpenShot video to rotate it 90 degrees to the left to avoid getting a crook neck!) here are my notes:

Benefits of Cloudron

Quick and Easy to install and configure properly apps on your own domains
Easy to update
Easy to maintain
Secure by default
Control of your data - you know and decide where it is stored, and who and how the data can be accessed

Selfhosting = you deploy apps on servers of your choosing

DNS management and SSL certificates all sorted using LetsEncrypt

Convenience of SaaS with the control of a private cloud

• with a user experience similar to what people are now used to with their smart phones - everyone can easyily install apps from the app store and apps are automatically updated - we don't have to maintain our phones, it just works.

Smart phone -> Smart server

Full Email server built in!!!

Can easily move (or clone) apps from one domain to another

== Architecture: ==

Cloudron is install on the server
Apps are listed from Cloudron's App Store
Cloudron install the App (it's not something the App Store does)
Cloudron periodically checks for updates
Each Cloudron installation is independent and private. Cloudron don't have access to the server. They just provie the App Store list and provide updates for apps

== Anatomy of an app ==

2 parts:

1. Docker based packaging

• dependency management (differnt apps require different versions of PHP, Ruby, whatever Docker lets you package them all together in a single package)
• Static configuration (decision about whether to use nginx or apache etc are taken care of)
• Density (used to have to use one virtual machine for each app, with Docker you can have loads of differnt apps on one server in containers)

2. Mainfest file (information about the things the app needs to run)

• Addons: which database to use (MySql, Postgres, etc), Auth, Email
• Port bindings
• Version: Title, Icon, Description, Author

== App Store ==

Just like Google Play story, Appple app store

• its just a Distribution mechanism, doesn't have access to the server.
• holds manifest meta data (what apps needs to run)
• versioning information

All the app packages are open source, so you

(not relevant for selfhost.cloud but it handles Dynamic DNS for people hosting at home)

== The Platform (the bit that is running on your server) ==

Very similar to Heroku - you can give it code and it'll run it for you

• Each addon is a micro-service (and can say, this app needs mysql server, this app needs to be able to send email etc)
• Addon access credentials as env var (environment variables)

(ask Girish for the slides)

All the addons (databases, email etc) are running in their own Docker containers and operate like mirco-services, you can ask them to create/ destroy databases etc.

== App Lifecycle ==

• Install
  -- Configure DNS
  -- Downloads docker image and manifest file from the Cloudron app store
  -- Sets up addons
  -- Logrotate, Collect (stats about the app, how much CPU, memory it using etc), Firewall
  -- Runs the docker container
  --- Dynamic configuration (giving the app db credentials, SMTP credentials to send email, e.g. for WordPress it creates the wp-config.php file with all the relevant credentials)
  -- Gets SSL certificates from LetsEncrypt (and set-up reverse proxy, i.e. when blog.domain.com is visited forward to this container)

• Updates
  -- Read only and stateless app containers (all apps are read only - apps cannot write to their file system, if they could then users could add all sorts of random files and it wouldn't be possible to update smoothly, the code cannot be modified. This means when there is an update we can just throw away the old container and get in the new container. So where does the app write stuff it needs to? We let the app write in 3 locations 1) /tmp for temporary files 2) /run which contains runtime files which an app needs to communication across various processes, 3) app/data/ where the app will put images, files uploaded etc - everything in app data is part of the backup, /tmp and /run is not backed up)
  -- Rolling updates
  -- Signed releases
  -- Selenium based tests (test that everything actually still works)

== Maintainence ==

• Backups
  -- Per app backups (means you can roll back just that app instead of all the apps on the server, like would happen if you just rolled back to the server snapshot).
  -- Backup only addon data (don't need to back up the docker container because it's read only, nothing has change, only need to back up databases etc, plus the /app/data directory)
  -- Apps can be trivially cloned and rolled back
  -- Can be stored offsite to S3, DO Spaces, etc

• Alerts
  -- Email notifications

== Security ==

• Turn-key security
• HTTPS only
• SSL, HSTS
• App isolation and sandboxed (apps can only access their databases etc, not the other databases etc)
• Rate limits, Activity logs (built in standard security practices e.g. Rate limtes: if people try to login too quickly, login from a new device etc, Activity logs: everything that happening on the system, who is doing what)
• Signed releases
• More info at https://docs.cloudron.io/security/

Who are Cloudron customers:

20-30% individuals
20-30% tech startups
universities (but they have differnt pricing structures)
very popular with ngos and cooperatives, in Europe (France, Germany) - ha! Girish described co-ops as new! hehe
also resellers and hosting providers

@thetomester13 a slight red flag re our Selfhost.Cloud plans - Cloudron started off providing managed Cloudron, but users wanted to install it on their own servers - still, I think they are likely still plenty of people who would like it as a service too. Just like people still sometimes pay me to create selfhosted WordPress sites for them and still use WordPress.com even those self-hosting WordPress is really easy.

@girish do you still have the slides you used in the talk? could you perhaps share them here? (even though I've more or less transcribed them plus added a load of stuff you said that isn't on the slides) Thanks!

[girish]

@jdaviescoates I will probably upload it in some place with a memorable URL later but these are the slides - https://files.cloudron.io/s/y6NinTqKdFHaBGi . Thanks for the elaborate notes!

[marcusquinn]

Adding a note the convo without completely reading but if there's effort to build a video library to promote and educate on features, I'd suggest aiming to segment into 2-5min videos per topic, attention-spans online aren't long enough for 30 min tutorials for the average prospect.

[jdaviescoates]

@marcusquinn and to add to that, any video that's going to be shared on socials (where most people discover video) really needs to be very clear what it's about and have a hook in the first 10 seconds (most videos aren't watch past that - if a video doesn't grab you in the first 10 seconds, you don't watch it).

[jdaviescoates]

@girish thanks!

[plusone-nick]

@jdaviescoates I have been messing around with a few "intro to cloudron" scripts, testing out OBS backdrops and mic settings. Will share a draft when I have a legit one compiled =] Hopefully sooner than later lol

[jdaviescoates]

@plusone-nick nice, looking forward to seeing it, thanks

[nebulon]

@jdaviescoates I rotated the video now and uploaded it to https://videos.cloudron.io/videos/watch/79a34d05-a60b-4ec3-9327-fd736016494c

[robi]

@jdaviescoates nicely done.

What is the file size?

Can you share with me the edited video, so I can post it to our new BayLISA.org YT channel?

Perhaps over a FilePizza link or whatever works for you.

[robi]

@nebulon ooh, thank you, just saw this! I'll see if we can use this one.

[jdaviescoates]

@robi yeah I think @nebulon has done a better job of rotating than I did. I guess he probably just used some command line tools? My file ended up over 3gb and had black space all around the edges.

[nebulon]

Indeed fyi I used:

ffmpeg -i BayLISA-Cloudron.mp4 -vf "transpose=2" BayLISA-Cloudron-rotated.mp4

the value 2 here is for 90° counter clock-wise, it took me 2 attempts to get the right value...happens when you don't read the docs

[robi]

@nebulon How big was the file?