What's coming in Cloudron 8.0

girish

Most of the apps are updated to the new base image, yay! That was a lot of work but atleast we can be confident that the apps work with the latest databases. So, we will push out a 6.2 release very soon with the following:

Move apps to new base image v3 - See https://forum.cloudron.io/topic/4366/docker-base-image-3-0
Database upgrades - MySQL 8, Postgres 12, Redis 5, Mongo DB 4.2.
Update node to 14.15.4
Update docker to 20.10.3
Make it possible to restore without updating DNS. This is useful if you want to test out cloudron backups.
Add a way to manage apps <-> groups better. Currently, it's hard to get a grip on what apps a user/group has access to easily (maybe add filters as well).
TLS addon (for DoT in AdguardHome)

girish

Just finished up on the "Dry run" feature for Cloudron restore. You can use this to test your backups, test migrate to another server to see how well the server performs (for example, when you switch VPS) etc.

The way it works is :

you select this "Dry run" checkbox. When enabled, it will restore as always except it won't update the DNS.
Before you click 'Restore', add an entry in /etc/hosts for my.domain.com to this new server's IP. You probably also want to add entries for your app domains if you want to test them.
Once restored, and you feel happy, you can "switch over" to this new server, using the new sync DNS feature. This is in the Domains view.

ruihildt

That's great!

Will you potentially lose some data during the backup/restore time that won't be carried over to the new one if you're running "dynamic" apps (like a forum)?

imc67

@ruihildt I was thinking the same! For example emails getting in to the old server will be lost I guess.

girish

@ruihildt yes, any data generated between the switch over is lost.

imc67

@girish wouldn't it be a good idea to have something like a "pause switch" all it does is (don't know technically):

"freeze" all the containers,
then make a backup,
migrate to new server,
do some testing and
switch DNS
finally "un-freeze / un-pause" all the containers on the new server (leaving the old one freezed)

Between 1 and 6 you can show the now already available and customizable custom_pages and incoming mail is already buffered at the sender.

d19dotca

@imc67 This is easy to do manually in the meantime, but yes I agree it'd be awesome to have this process sort of automated. For me it's mostly just email I focus on that's likely to change between backup and restore, so I do a full system backup and once it gets to the "box" part of the backup I disable the mail container so that it won't receive mail after the backup. Helps a lot. Of course doing this if you have clients relying heavily on email like I do then it means you have to do this in the middle of the night though lol. The ability to automate this process would be awesome.

girish

@imc67 Ah yes, we used to call this "retire" back in the managed hosting days We used to show a page that the server is migrating. Incidentally, I just removed the retire script last night! It's really just systemctl stop docker box but of course these days you will see a "app is down page" instead of a better "migration in progress" page.

I have created https://git.cloudron.io/cloudron/box/-/issues/774 but it won't make it to this release.

d19dotca

I installed and ran 6.2 this evening. So far it's looking great. Loving the changes and improvements. Especially happy with the 'missing backup' cleanup task So far so good! Thanks for all the hard work you guys!

One word of caution for anyone who hasn't upgraded to 6.2 (pre-release) yet... the downtime experienced during this upgrade was significantly higher than normal. Usually upgrades of Cloudron have practically no noticeable downtime or if it does it's only for a couple of minutes, but this time the upgrade took around 20-25 minutes before everything was running again which I'm sure (based on the logs) was because of the major version changes for MySQL, Postgres, Redis, etc. So just be prepared for a larger downtime during this upgrade.

Edit: I may have spoken too soon with regards to backups... running into two different issues with it now after upgrade to 6.2.

girish

I can confirm the bug. Those on OVH object storage for backups should not update to 6.2. Note that only the backup entries in the local database are removed, the backups itself are intact in OVH Storage. However, this means that Cloudron won't "clean up" the backups in remote storage properly since it has lost track of them.

Fix is coming in 6.2.1.

humptydumpty

@girish I closed the upgrade notification by mistake. What's the changelog for v6.2.1?

Edit: Nevermind! I found it in the "Event Log".

    "version": "6.2.1",
    "changelog": [
      "ovh: object storage URL has changed from s3 to storage subdomain",
      "ionos: add profit bricks object storage",
      "update node to 14.15.4",
      "update docker to 20.10.3",
      "new base image 3.0.0",
      "postgresql updated to 12.5",
      "redis updated to 5.0.7",
      "dovecot updated to 2.3.7",
      "proxyAuth: fix docker UA detection",
      "registry config: add UI to disable it",
      "update solr to 8.8.1",
      "firewall: fix issue where script errored when having more than 15 wl/bl ports",
      "If groups are used, do not allow app installation without choosing the access settings",
      "tls addon",
      "Do not overwrite existing DMARC record",
      "Sync dns records",
      "Dry run restore",
      "linode: show cloudron is installing when user SSHs",
      "mysql: disable bin logs",
      "Show cancel task button if task is still running after 2 minutes",
      "filemanager: fix various bugs involving file names with spaces",
      "Change Referrer-policy default to 'same-origin'",
      "rsync: preserve and restore symlinks",
      "Clean up backups function now removes missing backups",
      "Avoid updown notifications on full restore",
      "Add retries to downloader logic in installer"

@p44 Multihost support sounds amazing. It'll make Cloudron unstoppable I think.

p44

@af

@A Former User said in What's coming in Cloudron 7.0:

@p44 Multihost support sounds amazing. It'll make Cloudron unstoppable I think.

Totally agree on this point!! I'm sure Cloudron team will do an amazing work!

girish

We started rolling out 6.2 over the last week, I think it will take until mid next week for the full roll out. As a heads up, there are two known regressions:

Renew all certs button does not work in the Domains view.
Linode Object Storage backups does not work . Trying to work out a fix upstream - https://github.com/aws/aws-sdk-js/issues/3404

humptydumpty

@girish I was about to post a "bug" about certs. One of my domains failed to renew 13 hours ago. I received an email from my CR about it.

{   "domain": "www.mydomain.com",   "errorMessage": "Unexpected status: invalid" }

I checked the logs and here's the timeline.
13 hours ago - certificate renewal failed
12 hours ago - certificate renewal succeeded
1 hour ago - certificate renewal succeeded (it ran it again?)

{
  "domain": "mydomain.com",
  "errorMessage": ""
}

Checking the cert on the site itself, shows that it did renew it successfully.

Edit: BTW, this is the same domain that I receive random messages saying that DKIM records aren't set properly but I know they are. The message comes back up a few days later. DNS manually controlled for this domain in CR.

girish

We will make another pit stop at Cloudron 6.3 before Cloudron 7 - https://forum.cloudron.io/topic/4723/what-s-coming-in-cloudron-6-3

mdreira

I know you already have a lot of work, but if the ability to have alias domains could be added to this version, it would be incredible!!!!

I am needing it.

avatar1024

I wonder if there any scope for implementing the restic backend for backups in cloudron 8.0?
See this thread https://forum.cloudron.io/topic/1575/backup-improvements-restic-backend

mdreira

I wanted to ask something about multihosting in relation to Cloudron licenses.

I will give you an example: the main server has a Cloudron premium license. Two more servers have a free Cloudron license.

Could I control the applications on the free servers from the main server? Domains, apps, backups, emails...

girish

I wonder how valid/relevant this post is now, since it's like a year old And all the features except multi-host are now implemented.

avatar1024

@girish true that it's pretty hard for this post to keep up with your coding speed

Maybe we should start a new clean one and you can let us know what you think on some of the newer ideas that were proposed (e.g. this one on my side)

Well done for you amazing work! (and the friendly support I have to say!)

girish

@avatar1024 Ah yes, I will add it to the 7.2 list. We are rewriting the backup code to use async/await, it will make it easier to integrate newer backends like restic in the release after that. In the past, I was a bit wary of adding more backends but I wasn't sure how stable they were. But it seems, storage players like https://www.rsync.net/ support them, so they are probably quite stable.

mdreira

@mdreira said in What's coming in Cloudron 8.0:

I wanted to ask something about multihosting in relation to Cloudron licenses.

I will give you an example: the main server has a Cloudron premium license. Two more servers have a free Cloudron license.

Could I control the applications on the free servers from the main server? Domains, apps, backups, emails...

@girish Can you tell me something about this I wrote one month along? Thank you

MooCloud_Matt

@avatar1024

@avatar1024 said in What's coming in Cloudron 8.0:

(e.g. this one on my side)

Same here, restic support could be a great idea for backups as an alternative to rsync.

girish

@mdreira Currently, that is not possible. This is what "multi-host" cloudron intends to solve. It will take us a while to get there.

allanbowe

We would love to see the following:

TURN server support for jitsi (to enable enterprise calls over 443)
Well Known support - to enable arbitrary well-known configs (not just a predefined list)

robi

I think the OP needs an update with the 7.x series of changes and more 8.x features.

/cc @staff

matbrgz

I'm wating anciosly the addon for rabbitmq

girish

@matbrgz Given that very no app needs rabbitmq at this point, it's easiest to integrate this into the app itself using supervisor.

robi

For multi-host, it may be useful to have the DB data model as a local first concept, then sync with edge nodes.

VLCN does this as an open-source library for state management and sync.

VLCN augments SQLite, giving it the power to merge changes from other peers, notify your application of changes, and traverse remote datasets.
https://vlcn.io
https://github.com/vlcn-io/cr-sqlite

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

What's coming in Cloudron 8.0