What's coming in Cloudron 9
-
G girish forked this topic on
-
I don't see much in terms of new networking features - is there anything on the roadmap, for example multiple NIC support?
https://forum.cloudron.io/topic/7839/more-than-1-network-nic-bind-container-to-networks
(would be a great way to separate local apps like home assistant or immich from externally reachable apps, using just the GUI)Looking forward to the new backup capabilities, thanks! : D
-
Since many IPv6/PTR issues have been reported, I revisited the code to double-check.
I found two biggish bugs:
- There was a typo in the IPv4/IPv6 caching code
Because of this IPv6 will sometimes be returned as undefined - On versions less than Ubuntu 24, unbound was configured to use IPv6 . Zen SpamHaus is not replying IPv6 queries for most of the public VPS providers . I made a fix to make unbound use IPv4 to query SpamHaus.
Finally, I also added IPv6 DNSBL checks . Also, double checked that SPF record "a:" includes AAAA .
I am hoping this helps the situation. If not, we can add a flag in 9.1 to make the mail server not use IPv6 at all.
@girish said in What's coming in Cloudron 9:
Since many IPv6/PTR issues have been reported, I revisited the code to double-check.
I found two biggish bugs:
- There was a typo in the IPv4/IPv6 caching code Because of this IPv6 will sometimes be returned as undefined
- On versions less than Ubuntu 24, unbound was configured to use IPv6 . Zen SpamHaus is not replying IPv6 queries for most of the public VPS providers . I made a fix to make unbound use IPv4 to query SpamHaus.
Finally, I also added IPv6 DNSBL checks . Also, double checked that SPF record "a:" includes AAAA .
I am hoping this helps the situation. If not, we can add a flag in 9.1 to make the mail server not use IPv6 at all.
With some much needed Mail improvements, I just wanted to raise this one to your radar in case this can make it in for 9.0 at all. I’m surprised this isn’t making bigger headaches for people, likely just a particular set of circumstances, but I think this is an important fix or improvement to include if we can.
- There was a typo in the IPv4/IPv6 caching code
-
Also, still don't have an exact date for Cloudron 9 but the testing has been going on very well. I am hoping mid July we can publish the unstable.
@girish said in What's coming in Cloudron 9:
Also, still don't have an exact date for Cloudron 9 but the testing has been going on very well. I am hoping mid July we can publish the unstable.
Hi Girish. How's it going?
-
@girish said in What's coming in Cloudron 9:
Also, still don't have an exact date for Cloudron 9 but the testing has been going on very well. I am hoping mid July we can publish the unstable.
Hi Girish. How's it going?
@humptydumpty Getting there... Last week we landed the support for multiple backup destinations with independent backup schedules. Backups also now link internally to the backup destination. This way, if you delete a backup destination, it's clear that Cloudron has lost track of backups that were made in that destination i.e those backup entries are removed and are not listed in the UI (unlike now where they linger but it's actually not possible to restore).
Currently, working on adding integrity checks and better progress. With that we are done.
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
@girish said in What's coming in Cloudron 9:
This is a bit complicated because you have to download the backup to check the integrity...
Why not have a simpler level 1 integrity light check that is online only, with the file sizes and any files one doesn't need to download for example.
The level 2 deep check is to download and sift through all files.
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
@girish said in What's coming in Cloudron 9:
This is a bit complicated because you have to download the backup to check the integrity...
@robi said in What's coming in Cloudron 9:
Why not have a simpler level 1 integrity light check that is online only, with the file sizes and any files one doesn't need to download for example.
Yeah, couldn't Level 1 "Check integrity" just essentially be "do the hashes match"? (although presumably they always will otherwise the backup wouldn't be marked as having successfully completed?
)And a Level 2 "Full Integrity Check" (this will take a Long Time as it requires downloading the full back-up to ensure..."
Either way, all sounds like great progress, thanks!
-
The idea was to check for bitrot and give a good feel about the backup. Just checking the sizes doesn't mean much. AFAIK, file metadata is also stored in different sectors of the disk compared to the real data. So accessing file sizes and them matching doesn't mean much. (For tgz, there is also only one file size to check).
-
@CBCUN Any minute now..
-
I am sure you're working heavily on version 9, but just wanted to check-in to see if there was any ETA when we can use this on our existing Cloudron servers... just really eager & excited to see the many backup and UI improvements.
Can't wait for it. Good luck with it all! -
I believe there is good hope the release would land soon
...
I keep track of the status for milestone 9.0 on a daily basis (thanks to changedetection) and yesterday they closed the last tickets attached to milestone 9.0 https://git.cloudron.io/platform/box/-/milestones/84#tab-issues and until there is anything else hidden from the dashboard above, I hope it's for the coming days. But hey, I'm only a customer so of course I want all the best for yesterday xD -
Today was a big day internally
After many many months of work, we have finally tagged and published 9.0.0 . Sorry to be a wet blanket though - this is currently not available for new installations or even for an update.Our plan is something like this:
- We have currently upgraded our servers to 9.0.0. Since issues only appear in production and when running for a while, we will monitor it closely over the weekend.
- From next week, we will set 9.0.0 as the default for new installations. i.e
cloudron-setupwill install 9.0.0 by default. - After another week, we will provide it as unstable update.
- After that, we can roll out slowly.
There are still many minor bugs, it's listed here - https://git.cloudron.io/platform/box/-/issues/848 . I think by the time it is available even as unstable update, it will be 9.0.1 and the obvious issues will be fixed.
Now, if you really really are impatient and want to play around , you can install it on a fresh server using
cloudron-setup --version 9.0.0. We have changed the subscription flow in Cloudron 9, you don't even need a Cloudron.io account for the free plan / test run.edit: made a new thread to report bugs - https://forum.cloudron.io/topic/14359/cloudron-9.0-beta-bug-reports
-
Today was a big day internally
After many many months of work, we have finally tagged and published 9.0.0 . Sorry to be a wet blanket though - this is currently not available for new installations or even for an update.Our plan is something like this:
- We have currently upgraded our servers to 9.0.0. Since issues only appear in production and when running for a while, we will monitor it closely over the weekend.
- From next week, we will set 9.0.0 as the default for new installations. i.e
cloudron-setupwill install 9.0.0 by default. - After another week, we will provide it as unstable update.
- After that, we can roll out slowly.
There are still many minor bugs, it's listed here - https://git.cloudron.io/platform/box/-/issues/848 . I think by the time it is available even as unstable update, it will be 9.0.1 and the obvious issues will be fixed.
Now, if you really really are impatient and want to play around , you can install it on a fresh server using
cloudron-setup --version 9.0.0. We have changed the subscription flow in Cloudron 9, you don't even need a Cloudron.io account for the free plan / test run.edit: made a new thread to report bugs - https://forum.cloudron.io/topic/14359/cloudron-9.0-beta-bug-reports
@girish excellent stuff, thanks for the update! (and all the work!)
From my point of view right now, all we need now is an ElasticSearch addon/ app so we can have full text search in Nextcloud and we'll be rocking!
