What's coming in Cloudron 9
-
Since many IPv6/PTR issues have been reported, I revisited the code to double-check.
I found two biggish bugs:
- There was a typo in the IPv4/IPv6 caching code
Because of this IPv6 will sometimes be returned as undefined - On versions less than Ubuntu 24, unbound was configured to use IPv6 . Zen SpamHaus is not replying IPv6 queries for most of the public VPS providers . I made a fix to make unbound use IPv4 to query SpamHaus.
Finally, I also added IPv6 DNSBL checks . Also, double checked that SPF record "a:" includes AAAA .
I am hoping this helps the situation. If not, we can add a flag in 9.1 to make the mail server not use IPv6 at all.
@girish said in What's coming in Cloudron 9:
Since many IPv6/PTR issues have been reported, I revisited the code to double-check.
I found two biggish bugs:
- There was a typo in the IPv4/IPv6 caching code Because of this IPv6 will sometimes be returned as undefined
- On versions less than Ubuntu 24, unbound was configured to use IPv6 . Zen SpamHaus is not replying IPv6 queries for most of the public VPS providers . I made a fix to make unbound use IPv4 to query SpamHaus.
Finally, I also added IPv6 DNSBL checks . Also, double checked that SPF record "a:" includes AAAA .
I am hoping this helps the situation. If not, we can add a flag in 9.1 to make the mail server not use IPv6 at all.
With some much needed Mail improvements, I just wanted to raise this one to your radar in case this can make it in for 9.0 at all. I’m surprised this isn’t making bigger headaches for people, likely just a particular set of circumstances, but I think this is an important fix or improvement to include if we can.
- There was a typo in the IPv4/IPv6 caching code
-
Also, still don't have an exact date for Cloudron 9 but the testing has been going on very well. I am hoping mid July we can publish the unstable.
@girish said in What's coming in Cloudron 9:
Also, still don't have an exact date for Cloudron 9 but the testing has been going on very well. I am hoping mid July we can publish the unstable.
Hi Girish. How's it going?
-
@girish said in What's coming in Cloudron 9:
Also, still don't have an exact date for Cloudron 9 but the testing has been going on very well. I am hoping mid July we can publish the unstable.
Hi Girish. How's it going?
@humptydumpty Getting there... Last week we landed the support for multiple backup destinations with independent backup schedules. Backups also now link internally to the backup destination. This way, if you delete a backup destination, it's clear that Cloudron has lost track of backups that were made in that destination i.e those backup entries are removed and are not listed in the UI (unlike now where they linger but it's actually not possible to restore).
Currently, working on adding integrity checks and better progress. With that we are done.
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
@girish said in What's coming in Cloudron 9:
This is a bit complicated because you have to download the backup to check the integrity...
Why not have a simpler level 1 integrity light check that is online only, with the file sizes and any files one doesn't need to download for example.
The level 2 deep check is to download and sift through all files.
-
An update on backup integrity: integrity information is now stored in
.backupinfofiles in the remote along side the backups . The file contains sha256 of the backup . For tgz, it's a single hash. For rsync, it contains individual hashes. Further, the backupinfo file is signed using a private key and the signature is stored in Cloudron database. Using this setup, we can verify the authenticity and integrity of backupinfo file (i.e it was created by the Cloudron backup system and was not altered) and we can also check the backups itself are not corrupt using the sha256.While implementing this, I have also added fileCount and size to each of the backup entries. So, you can get an idea of how many files are in the backup and the total aggregated size of an individual backup.
Currently, working on the integrity verifier i.e you can click some button to say "Check integrity" and it will verify the integrity of the backup. This is a bit complicated because you have to download the backup to check the integrity...
@girish said in What's coming in Cloudron 9:
This is a bit complicated because you have to download the backup to check the integrity...
@robi said in What's coming in Cloudron 9:
Why not have a simpler level 1 integrity light check that is online only, with the file sizes and any files one doesn't need to download for example.
Yeah, couldn't Level 1 "Check integrity" just essentially be "do the hashes match"? (although presumably they always will otherwise the backup wouldn't be marked as having successfully completed?
)And a Level 2 "Full Integrity Check" (this will take a Long Time as it requires downloading the full back-up to ensure..."
Either way, all sounds like great progress, thanks!
-
The idea was to check for bitrot and give a good feel about the backup. Just checking the sizes doesn't mean much. AFAIK, file metadata is also stored in different sectors of the disk compared to the real data. So accessing file sizes and them matching doesn't mean much. (For tgz, there is also only one file size to check).
