What's coming in Cloudron 6.3

girish

Before Cloudron 7, we need some more work to make the single server install secure. For this reason, we will spend some time first with the following:

(Security) - Inform users about new browser/IP logins.
(Security) - Better email monitoring/visibility for admins. @d19dotca raised many important posts and there's also existing ones. We have to read the posts in more detail and discuss internally before we give more details on what we plan to do here. (moved to next release)
(Security) - Add a way to secure/limit access to specific apps and dashboard. For example, a set of apps are public and the rest are only accessible via wireguard/openvpn. This combined with mandatory 2FA for dashboard will make good security. (moved to next release)
Reduce/remove some notifications. It seems a bit noisy.
Fix email situation for Go apps like Statping, Commento that are having trouble sending mails via our email server.
Make email setup inside apps optional. This will make it possible to configure specific apps to use some external service for mail delivery directly and the Cloudron package won't touch their mail settings.
Volumes - make mounting easier by automating fstab/exports entries
~~Move TURN server to port 443.~~ (moved to next release)
As a pre-requisite for Cloudron 7 multi-host feature, we have to move file system data into the database. Much grunt work to be done here.
Vultr DNS
Vultr Object Storage

girish

@girish said in What's coming in Cloudron 6.3:

Make email setup inside apps optional. This will make it possible to configure specific apps to use some external service for mail delivery directly and the Cloudron package won't touch their mail settings.

This one is implemented now in the Email view. The app package has to explicitly say whether it supports this feature or not using the optional flag to the sendmail addon.

mdreira

@girish Good job!

Now with Mautic we have to use Amazon SES, but if all these improvements are implemented in the Cloudron mail server, we can forget about Amazon.

imc67

@girish said in What's coming in Cloudron 6.3:

(Security) - Inform users about new browser/IP logins.

I was already thinking for days to express my concerns about security on Cloudron, so happy to see some improvements.

But also I would like to invite @girish and @nebulon to get inspiration from the Wordpress plug-in called Wordfence. Wordfence imho needs to be installed by default in Wordpress and they have excellent security measures and management (also in the free version), here some I would like to see on Cloudron:

don't inform users about new browser/IP/location logins but the Admin, users usually don't even know what an IP is. In Wordfence there is a setting to inform the Admin (via email and log) about new IP/browser/location of either admins and/or users
create a separate login log GUI with successful and especially unsuccessful login attempts and also on the "individual log record" the possibility to block the, either misused username or login IP for x-time or forever
when you notice login attempts of non-existing users, also create the possibility to create a blocking list of non-existing usernames that are commonly misused and block them forever
extend user setting: https://forum.cloudron.io/post/24708
IP logging in apps with real IP, in https://forum.cloudron.io/post/24706 it's solved but many other apps like Freescout it's still the docker IP
GEO blocking of countries: https://forum.cloudron.io/post/19901
make the login rate limiting configurable: https://docs.cloudron.io/security/#rate-limits
make the activity log also log every LDAP login (attempt), not only replace the LDAP login log record by the last login attempt

d19dotca

@imc67 I definitely agree, lots of areas for improvements.

I do have one item of your post I wanted to share my two cents on though...

Wordfence imho needs to be installed by default in Wordpress

This is just my personal view so take it with a grain of salt of course, I don't think there's really any right or wrong way to it. My two cents...

I have the belief that we should aim to keep all apps (WordPress included) about as close to default as the developer intended for the app, leaving Cloudron to just handle the default user and mail config for the app, etc. I think it's a bit of a slippery slope to add in all these extras regardless of how important they may be for certain use-cases, because the line needs to be drawn somewhere of course and deciding where that line is isn't particularly clear. In such a case, I'd ere on the side of "keep it as close to default as possible as intended by the developer" to stay free of any tightrope walking so-to-speak. haha.

For WordPress in particular, there's practically hundreds of thousands of plugins available, some of which are "default installs" in my environments where I have a template setup with the ones I use all the time for example, but I would never want to force my defaults on others because what works for me or the plugins I prefer to use may not work or be preferred by other users. Security plugins are one area where there's a ton of them, and there was a similar discussion not too long ago I had regarding a suggestion for including a caching plugin by default too.

And that's kind of the slippery slope I am referring too... what is the criteria for when a "default plugin suggestion" gets approved, and when would one get denied?

I just really want to see apps be as close to default as possible. But of course that's just my two cents. I'm sure many might disagree with me. haha.

I totally agree with the rest of your suggestions though, I would love to see the improvements for additional security in the Cloudron server itself. I completely forgot about the GEO-blocking request for countries, that'd be pretty great to have!

mehdi

@girish said in What's coming in Cloudron 6.3:

As a pre-requisite for Cloudron 7 multi-host feature, we have to move file system data into the database. Much grunt work to be done here.

I am not sure I understand what you mean by that. Could you expand a bit on this please?

imc67

@d19dotca said in What's coming in Cloudron 6.3:

I totally agree with the rest of your suggestions

it wasn’t my intention at all to suggest to add Wordfence by default in the Cloudron package as indeed everyone has to decide themselves. Personally I install it in every Wordpress site I manage.

jdaviescoates

@imc67 said in What's coming in Cloudron 6.3:

Personally I install it in every Wordpress site I manage.

Me too.

ruihildt

Great you're spending time on notifications and email!

I like the idea of a centralized place where you can configure which type of notifications you get(Platform update available/completed, app update available/completed, app error, backup error/completed, etc.), and where(email, dashboard, webhook, etc.).

A bit like Facebook, Twitter have.

d19dotca

@imc67 said in What's coming in Cloudron 6.3:

it wasn’t my intention at all to suggest to add Wordfence by default in the Cloudron package as indeed everyone has to decide themselves. Personally I install it in every Wordpress site I manage.

Oh my bad, sorry I misunderstood the intention there.

girish

@girish said in What's coming in Cloudron 6.3:

Fix email situation for Go apps like Statping, Commento that are having trouble sending mails via our email server.

our patch got merged upstream for this - https://github.com/haraka/Haraka/pull/2940

mdreira

@girish said in What's coming in Cloudron 6.3:

@girish said in What's coming in Cloudron 6.3:

Make email setup inside apps optional. This will make it possible to configure specific apps to use some external service for mail delivery directly and the Cloudron package won't touch their mail settings.

This one is implemented now in the Email view. The app package has to explicitly say whether it supports this feature or not using the optional flag to the sendmail addon.

@girish In 6.2.7 update this should already be implemented, right? I thought I read it in the changelog.

However I have not yet seen this option in the applications.

girish

@mdreira the feature is only in 6.3 (not released) and not 6.2.

mdreira

@girish Thank you. I am looking forward to that day! I need this feature very much.

Do you have any planned date when update 6.3 will be released?

girish

@mdreira It doesn't have a release date yet. But it should be out end of april for sure.

girish

There's an option now for admins to reset user's 2FA:

girish

Mailboxes and lists can be individually set as active/inactive. When inactive, you cannot login to the mailbox and mails will bounce:

robi

@girish said in What's coming in Cloudron 6.3:

There's an option now for admins to reset user's 2FA:

Perhaps change the button to say "Reset 2FA"

d19dotca

Just wanted to check in and see how 6.3 is coming along.

Any ETA by chance? I'm super excited for these email improvements many of us have been requesting, particularly the DNSBL checks; greylisting; blocklist & whitelist auto-updating/DNSWL; email autoexpunge; and not forwarding spam to mailing lists. I know that's a lot, lol.

I know many of them came from me, haha, so if you want to discuss any of them or want clarification on the requests, I'd be happy to help offer guidance or clarification.

girish

@d19dotca Thanks for checking We haven't gotten to the email part yet. I am fixing up the notification issues. Once I do that, I want to look into the wireguard/VPN thing before I get into email. @nebulon is working on the login history and I think that is mostly done. He is also working on the volume mounting (i.e will automatically setup fstab entries).

I don't have an ETA, will have a better idea next week. It's been a bit slow this week. I had my pfeizer vaccine, yay and now the sideeffects are gone, so I can go back to being productive

mdreira

I'm waiting like gold for this update, especially because Cloudron Mail is changing Mautic email configuration all time.

The Amazon SES-API and queue configuration that I have within Mautic is misconfigured every time the application restarts, updates, recovers ... it's frustrating.

girish

New browser login locations is implemented. This is only for dashboard logins and not for LDAP login (because many apps send mails by themselves).

fbartels

@girish said in What's coming in Cloudron 6.3:

This is only for dashboard logins

Since this already covers the dashboard, is it automatically available for apps using proxyauth as well?

nebulon

@fbartels we could do that for the auth proxy as well, but right now it is not. For other apps using LDAP this will be a bit harder since the ldap server currently does not see the upstream user-agent or IP.

fbartels

@nebulon ah, I kind of thought that the proxyauth would use the very same mechanisms of the dashboard for authentication. But the dashboard being mainly driven by an api with token authentication that of course does not have to be true.

Totally understand that the same is not possible for apps that use ldap under the surface.

girish

@girish said in What's coming in Cloudron 6.3:

As a pre-requisite for Cloudron 7 multi-host feature, we have to move file system data into the database. Much grunt work to be done here.

A big chunk of this landed today. Certificates also need to be migrated to the database, that should be done tomorrow. Essentially, from the next release, /home/yellowtent/boxdata will only contain mail server data and nothing else since everything has moved to the database. I will probably take this opportunity to separate box backups and mail backups. The box backup is going to be just the mysql dump and nothing else.

girish

@girish said in What's coming in Cloudron 6.3:

As a pre-requisite for Cloudron 7 multi-host feature, we have to move file system data into the database. Much grunt work to be done here.

This is now done! Now the boxdata only contains the mysqldump and email.

root@my:/home/yellowtent/boxdata# ls -l
total 900
-rw-r--r-- 1 yellowtent yellowtent 913492 May  7 06:00 box.mysqldump
drwxr-xr-x 9 yellowtent yellowtent   4096 May  4 07:34 mail

I am looking into moving mail as a separate backup just like an app. That way in future releases we can restore mail data (mailboxes) independently of box code just like apps.

girish

Pushed a change that makes the filenames of backups more readable. It's just app_<domain>_vVersion.

root@my:/var/backups/2021-05-09-000811-352# ls
app_test.cloudron.work_v1.1.3.tar.gz

girish

@girish said in What's coming in Cloudron 6.3:

Volumes - make mounting easier by automating fstab/exports entries

This is also mostly done. When adding a volume, you can choose the mount type

The current volumes are migrated as "no-op" mount type (as in, user managed the mount themselves). It shows the status of each volume as well:

One thing we decided to go with systemd mounts instead of /etc/fstab. This allows us to create mounts that have correct dependency with the unbound DNS server for CIFS and NFS mounts.

A similar mounting change will be done for the Backups view as well.

jdaviescoates

@girish said in What's coming in Cloudron 6.3:

One thing we decided to go with systemd mounts instead of /etc/fstab.

So what will happen to existing volumes that are already mounted using /etc/fstab?

nebulon

@jdaviescoates I think instead of coming up with migration code, which will be a bit messy given the fstab format to correctly parse in all circumstances, I think we will ignore those and ask the admin to reconfigure the volume via the UI once. That way the admin can test and validate timely.

jdaviescoates

@nebulon said in What's coming in Cloudron 6.3:

I think we will ignore those and ask the admin to reconfigure the volume via the UI once. That way the admin can test and validate timely.

Just to be clear, if the admin takes no action will existing volumes keep working?

nebulon

@jdaviescoates yes, the code does not touch existing mountpoints as such.

girish

We now show the ubuntu version is the settings view.

Additionally, there is now an alert for Ubuntu 16 users.

micmc

@girish AWESOME!

This looks wonderful! Loving this QoL changes.

d19dotca

@girish Please tell me 6.3 is coming down the pipe this week. I'm so eager for it! Desperately needing some of these email improvements, specifically the most urgent for me is the ability to not forward email on to mailing lists if it's been identified as spam. The limitations currently are impacting the trust of my mail server by other systems like Gmail which is rate limiting my emails now because so much spam is forwarded on to a couple of Gmail addresses via the mailing list functionality. At least they're not outright blocking me, but that'd be the next logical step that I want to avoid!

girish

@d19dotca it's unlikely this week, we just had a call yesterday and decided to release what we have right now. So, we have already started testing and running e2e. I will leave a note on the progress here. I moved the security+email features to the next immediate release. As for the specific issue you are facing wrt not forwarding spam, let me see if the fix for that is easy and include it in this release itself.

d19dotca

@girish Oh sure, that'd be good. So there's going to be a bug fix version then I presume with what's already been done so far (such as 6.2.9 maybe or still 6.3.0)? And then email + security will be added to something like 6.4 instead if the next release is still 6.3?

girish

@d19dotca what's next is 6.3. the email + security will be 6.4 or maybe even part of cloudron 7. we are actually very close to multi-host with the changes in 6.3 !

BTW, about the spam fix, it's easy to check if an email is marked as spam and drop it (i.e based on spamassassin flags). However, since we don't have a place to quarantine yet, you will only see a eventlog entry that it's gone. Is this OK for the moment till we implement a monitorable mail queue?

girish

As a final update to the mounting logic, it is also integrated into backups and restore logic.

scooke

@girish said in What's coming in Cloudron 6.3:

Pushed a change that makes the filenames of backups more readable. It's just app_<domain>_vVersion.
root@my:/var/backups/2021-05-09-000811-352# ls
app_test.cloudron.work_v1.1.3.tar.gz

Thank you!!

d19dotca

@girish said in What's coming in Cloudron 6.3:

BTW, about the spam fix, it's easy to check if an email is marked as spam and drop it (i.e based on spamassassin flags). However, since we don't have a place to quarantine yet, you will only see a eventlog entry that it's gone. Is this OK for the moment till we implement a monitorable mail queue?

I think that's fine for now, but would it be possible to only affect mailing lists perhaps (assuming this change is mostly just temporary and probably will only really be used by me for the time being). I ask because I think if I were to just drop all messages marked as spam from all mail delivery completely, it'd be a detriment to my hosted mailboxes. The only place I want to disable sending spam mail is for mailing lists alone, if possible. But either way I guess, better than nothing I need to ensure Gmail isn't going to block mail from my server completely just because the two addresses my mailing lists forward to are total magnets for spam.

subtlecourage

@girish said in What's coming in Cloudron 6.3:

Make email setup inside apps optional. This will make it possible to configure specific apps to use some external service for mail delivery directly and the Cloudron package won't touch their mail settings.
Volumes - make mounting easier by automating fstab/exports entries

The email feature is something I think will help my use caase.
The volume feature is freaking phenomenal.

You are all amazing!

girish

Now that Cloudron is in the Vultr marketplace, it seemed the right time to implement Vultr DNS.

girish

The last change remaining in 6.3 was related to notifications and most of it had landed yesterday.

Common issue is that a notification like a cert failure or restart required etc is essentially spammed to all admins and this causes great distress for all. So, we have now moved notifications from being per-user to be instead "system" level. The notification system is thus shared by the admin team and they can figure how to collaborate with other tools instead of Cloudron trying to have some sort of "read" indicator for each one of them.
Apart from failed backups and cert renewal failures, no emails are sent anymore. You have to visit the dashboard to see notifications since most of them are really not urgent or actionable.
Cert renewal notification + email is only raised if only 10 days are left to go for existing certificate to expire. The renewal itself will start 30 days in advance. So, we should see this very rarely from here on. Backup renewal notification will only be sent if 3 consecutive backups fail. This way we allow for "external services" to fail now and then without being too aggressive about notifying user.

There's more notes at https://git.cloudron.io/cloudron/box/-/commit/73917e95c9473ec8fc11216fd78607b0a256df7d . There's other minor visual changes as well, I will post a screenshot tomorrow once I complete the UI side.

girish

The going is slow... Having to support 3 ubuntu versions takes a lot of testing time The tests currently pass for ubuntu 18 and 20 but not for 16 yet. But we are close! Looks like failures are related to older versions of unbound not supporting sd_notify correctly.

d19dotca

@girish said in What's coming in Cloudron 6.3:

Having to support 3 ubuntu versions takes a lot of testing time

Maybe it's time to kill off support for Ubuntu 16?

girish

@d19dotca yes, from the next release, it won't be supported anymore. There is a notification to inform user to upgrade (I guess we should have had the foresight to do this the previous release itself..)

girish

The tests are passing . Now just a whole lot of manual testing next week.

d19dotca

Just thought I'd check-in on the status of 6.3.0.

girish

@d19dotca Manual testing of volume mounting revealed a lot of bugs, so we had to rework it a bit. The rework is done (as you can see from our git commits). I have started another e2e run just now.

Also, since we are now in the Vultr marketplace, we have gone ahead and added Vultr DNS and Vultr Object Storage as well in this release.

girish

Looks like we are all set. There's a few known issues which we will fix immediately - https://git.cloudron.io/cloudron/box/-/issues?milestone_title=6.3.1 . I will stage the release tomorrow morning and leave a note here.

girish

I pushed out 6.3.0. We hit a few migration issues, so it became 6.3.1. There are some minor issues being fixed, will be pushing 6.3.2 shortly. Please hold before you update to 6.3.

(to clarify, new installations get 6.3 but updates from 6.2->6.3 will give you a warning that it's unstable. please heed the warning ).

msbt

@girish what if some eager user already updated to 6.3.1, will those issues be resolved or will they run into problems?

girish

@msbt it should mostly be OK but there might be some regressions. Did you happen to hit any? Let me know so I can get it sorted out for 6.3.3 . So far, I haven't heard of any from the new installations and all the Cloudron we have access to seem to have updated OK.

msbt

@girish the only things that happened were the logo from the branding disappearing and the app-update icons were misplaced (top right corner ). After updating to 6.3.2 the arrows are back where they belong (although on the left side, but this is by design I reckon). The logo I just reuploaded. Other than that it all seemed fine

girish

@msbt those were the exact 2 regressions that were fixed there are some small issues in volumes which is getting fixed as well now in 6.3.3. the logo/branding is fixed in 6.3.2.

d19dotca

@girish - I see that the mailing lists aren't forwarding spam anymore and see it in the "Denied" categorized email logs which is great and helps a huge amount for many of my mailing lists. But... how do I overwrite this for certain mailing lists which are too sensitive to do spam filtering on? I don't see a way to modify this and one of the mailing lists are getting too many false-positives from one particular email I can't whitelist. Am I perhaps missing the checkbox to disable/enable the spam filtering for the mailing lists?

I have a bit of an urgent issue here that I need to resolve since I have a doctor who's not getting emails from a particular patient due to this. How can I resolve this?

Side note: It'd be awesome if the log entries showed which spam rules were triggered to mark it as spam, so that if it's a false-positive like it is then I can review and tweak as necessary. Also whitelisting emails and so many more things we really need in the email functionality. Please and thank you. I know the last few won't come until future versions, but in the meantime if I can somehow get around the mailing list spam filtering for a particular mailing list or if there's even a temporary way for me to whitelist/allowlist the email address being blocked, I'd really appreciate it.

robi

@d19dotca all for one? or one for all?

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

What's coming in Cloudron 6.3