What's coming in 5.6
-
Just one more sprint before 6.0. Mostly, it's mail related. Just trying to address some immediate needs in this release:
- Mail
- Make some of the existing settings configurable via the UI. Whitelist/blacklist, max message size, tls configuration etc.
- Make mail server name configurable. Instead of
my.domain.com
as email server, you can setup the name asmail.domain.com
, for example. - Autodiscover/Autoconfig support for email - https://git.cloudron.io/cloudron/box/-/issues/556 . I am not sure how easy/hard this is, but worth trying.
- Server side signature - If haraka allows this easily, we will add it.
Keep mail backups separate from box backups. This will make it easy to backup/restore emails separately.EDIT: decided it's a lot of work, postponed to some future release.
- Add a way to whitelist inbound ports. This will help apps like netdata, snmp monitors and other monitoring "agents".
- Add flag in manifest for reverse proxying to a port (for the OLS app).
Update MongoDB to 4.2- EDIT: this was not needed. The index length problem is solved by having smaller mongodb database names.- Hardware transcoding for emby/jellyfin (and in the future plex)
- Do not auto-update to pre-release version
- Allow configuring schedule of platform updates
Network usage graphs(See issue)- Optimize WP and Nextcloud installations
Anything small/obvious I left out?
- Mail
-
Any update on when we might could see network graphs per app?
-
@girish said in What's coming in 5.6:
Add flag in manifest for reverse proxying to a port (for the OLS app)
OLS app?
-
@Hillside502 Open Lite Speed - https://forum.cloudron.io/topic/2346/openlitespeed-wordpress (maybe it's not a well known acronym, I just made it up).
@doodlemania2 Added!
-
@girish Definitely looking forward to the first two mail features in particular!
Is it possible to expand on the "Keep mail backups separate from box backups" part? Is the intention here to restore just mail alone instead of needing to restore the whole server to restore mail? If so - that'd be awesome! Haven't needed that yet but I can imagine a few scenarios where that may happen in the future and make recovery much easier.
Keep up the hard work! Great job you guys!
-
@d19dotca Yes, the idea is to be able to independently backup and restore the mail stuff and also be able to set a location to it, just like any other app. Just like we have
app_
andbox
in backups, idea is now we will have amail_
in the backups directory (currently, it is inbox/mail
). -
A progress update on the release.
-
VAAPI (hardware transcoding on linux) support work well now. I had posted an update here - https://forum.cloudron.io/topic/3035/transcoding-update
-
We have decided against separating the mail and box backups for this release. There's some complications to do this properly (the mailbox and domain information is currently in the box code, we have to move this to the mail container. it's quite a bit of work and we feel this feature is not important at this point to justify the surgery).
-
Cloudron will not auto-update to pre-release version if you simply just checked for an update.
-
Scheduler: if you have a bunch of WP apps (especially unmanaged WP), you will see a lot of networking churn in dmesg and docker logs output. We have seen in a couple of customers who have over 100 WP installations, docker gets into some strange deadlock (iptables locking). To circumvent this, the cron job scheduler has been re-designed now so as to not create a new container for every cron run and also shares the networking namespace of the app container. This change requires no changes to apps.
-
Allow box auto update pattern to be configurable. The update schedule that in the Settings UI now applies to both the app and platform auto updates (previously, it only applied to app auto updates). This gives you more control on when the updates happen.
Phew, already changes are piling up, so will probably have to drop some of the planned features.
-
-
@girish Still hoping for this one to make it into Cloudron eventually...:=-)
-
@Mallewax Yes, I have added it in our 6.0 list now - https://forum.cloudron.io/topic/2760/what-s-coming-in-6-0 . In fact, I just moved to a new device and search not working reasonably is a bummer.
-
Nice work on the Mail server side of things!
Being able to set mail server subdomains to the more commonly used
mail.
will be a bonus.Thunderbird checks common subdomains as a second resort, although the extra DNS records for the standard autoconfig stuff would be better for more clients to autodetect.
I'm hoping that will help with email client auto-setups but anything you can do to also add the necessary records would be super handy:
-
@girish Maybe this is of help, too....
https://workaround.org/ispmail/buster/mail-client-auto-configuration/
-
@doodlemania2 It turns out collecting network information for each container is quite complicated. It's easy to collect the stats at a point in time (like docker stats does) but to do it over a time period, requires us to periodically track the container/process. I put some notes here - https://git.cloudron.io/cloudron/box/-/issues/734 . I have to postpone that task since it's quite complicated. If someone has better technical knowledge on this, please do leave a note on gitlab.
-
Thanks for the update good sir!
-
Any ETA on 5.6? Can't wait for these mail improvements.
-
For the email auto-configuration, for now I will just add docs on how to set this up manually. It's a bit complicated to automatically set all this up because outlooks requires SRV records (for which we have to write automation code for the DNS providers) and thunderbird/k9 requires setup of
.well-known
files. This has the same issue as mastodon where we can automate only the website is also on Cloudron. So, for now, I will put docs on how anyone can set this up manually. -
@will It can but it only works when the domain's web server is hosted on Cloudron. For example, cloudron.io website is hosted on a different server than cloudron.io email. The well-known has to be added on whichever server is hosting the website and not the email server.
-
@will I see what @girish is saying:
- One Cloudron managed server has Email hosted with it's various subdomains
- Another (maybe Cloudron managed) server has the root website domain.
- So the
.well-known
file needs hosting on the root domain server.
I've been thinking a fair amount recently about whether to
www.
the main website or not.Originally, I though, who still types
www.
?But, that misses the ability to manage it as just another subdomain, and have the main root domain 301 redirect to
www.
.Reasons for having
www.
? Well, this use-case could be one. CNAME load-balancing could be another.I'm thinking that Google does
www.
for their main domain and many other big players do too, so maybe they are right and my non-www.
thinking for succinct brevity was missing some of these other possibilities that usingwww.
enables.Might be an SEO risk changing large websites and potentially messing up the 301 redirects for all indexed URLs but I'm having a serious think about going back to
www.
given those using it probably know more than I do as to why it might be better. -
d19dotcareplied to marcusquinn on Aug 31, 2020, 6:32 PM last edited by d19dotca Aug 31, 2020, 6:39 PM
@marcusquinn For most people it doesn't matter if we use www or non-www, but I found this page many years ago which has made me stay consistent in using www over non-www, which you may find interesting too.
It mostly comes down to technicalities really (but this is probably a good example of one too): https://www.yes-www.org/
Or more specifically, this direct page for the reasons why to use www over non-www: https://www.yes-www.org/why-use-www/
With that said though, while the vast majority of major websites use www, some very large companies don't either, so there must be some valid reason or workaround they've found to any restrictions. Twitter is one that comes to mind, for example. That article actually mentions Twitter too. haha.
-
@d19dotca Very handy, thank you - you and they have me convinced now.
We have a load of Kubernetes stacks - but I feel they are overkill for most of your needs and the pods and replication has network latency, so I'm looking at more of a master-slave hot-swap high-availability strategy that will ultimately be cheaper, easier to understand and simpler to scale.
-
Another progress update. There is now a very basic firewall configuration UI to block IPs. You can use this to block connections to the mail server as well. In future releases, we will probably make this more dynamic. For example, just automatically block IPs for a specific period of time, if there are too many failed logged in attempts. We have to implement this across apps and the app logs are not too fail2ban friendly right now, so we have to see how we can do this.
There is also a mechanism to white list ports (will be documented) but there won't be a UI for this since I am not sure we should go about encouraging people to install things on Cloudron.
I think apart from some UI adjustments that @nebulon is making, we are set for the release!
-
@girish Just wanted to check for an update on this as I am really needing to store emails on a different location just like how apps work (my medical clinic clients are using email massively compared to before because of COVID-19 testing they've been doing the last few months, and the storage space usage is growing very quickly. I know you postponed the mail backup part where it was separated from box backups, but does this negatively impact the ability to store emails on an external disk too or is that feature still coming for storing emails on external disks?
-
@d19dotca said in What's coming in 5.6:
I know you postponed the mail backup part where it was separated from box backups, but does this negatively impact the ability to store emails on an external disk too or is that feature still coming for storing emails on external disks?
Yes, we postponed the splitting up of box data and mail data. But moving mail to a separate disk is already possible. Just that box data also goes with it. This should not be a problem for your situation. Essentially, the directory
/home/yellowtent/boxdata
(which contains the emails) can be moved anywhere you want. See https://cloudron.io/documentation/storage/#default-data-directory . Essentially, stop docker and cloudron, copy over all the stuff to new location, then just create a symlink to the new location.(Just make sure you have backups, just in case). Note that the external disk has to be ext4.
-
@girish Dang, that's too bad. Thank you for confirming though. I'll keep that process in mind as something I can do when the time comes then (which I suspect will be in about 2 months or earlier), but hoping there'll be an easier way to still have the box data on the local disk rather than needing to move it in its entirety to a new disk. Just as how the app locations can be changed and it automatically moves the data, it'd be great if Cloudron could do that part too without needing a manual intervention for it. Plus it'd save a little bit of cost to the admin as they wouldn't need to create such a large external disk for it all, just for the emails.
-
The 5.6 pre-release is out. Only update if you are truly eager. We will roll this update out very slowly over this week and the next.
If you hit any issues, please do open separate threads. The docs are still getting fixed up, so let me know if you find something hard to follow.
-
@girish said in What's coming in 5.6:
- Add a way to whitelist inbound ports. This will help apps like netdata, snmp monitors and other monitoring "agents".
- Add flag in manifest for reverse proxying to a port (for the OLS app).
Could I have more detail about these 2 features ? They look quite interesting !
-
@mehdi said in What's coming in 5.6:
Add a way to whitelist inbound ports. This will help apps like netdata, snmp monitors and other monitoring "agents".
https://docs.cloudron.io/networking/#whitelist-ports
As for the port proxying, it didn't make it to this release. It was meant for the OLS app that @MooCloud_Matt packaged. We have to try it for the next release.
-
@girish I just updated to 5.6.0 to test it out. Looks great so far! Only thing I'd suggest changing or fixing is the Max Message Size on the Email page, as it's default at 25 MB but when I go to change it, it goes in weird 10 MB intervals from 1 MB, so it's 21 MB, 31 MB, 41 MB, etc which are strange numbers to use. I'd suggest it either be configurable (i.e. a text box) or if it's on a slider then it should go up in 1 MB increments rather than 10 + 1.
-
Update to 5.6 failed with error but seems to work anyway.
Log:
Sep 12 09:34:03 box:tasks 4283: {"percent":70,"message":"Installing update"} Sep 12 09:34:03 box:shell update spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/update.sh /tmp/box-2509650539 Sep 12 09:34:03 box:shell update (stdout): Updating Cloudron with /tmp/box-2509650539 Sep 12 09:34:03 box:shell update (stdout): => reset service cloudron-updater status in case it failed Sep 12 09:34:03 box:shell update (stdout): Failed to reset failed state of unit cloudron-updater.service: Unit cloudron-updater.service is not loaded. Sep 12 09:34:03 box:shell update (stdout): => Run installer.sh as cloudron-updater. Sep 12 09:34:03 box:shell update (stdout): => starting service (ubuntu 18.04) cloudron-updater. see logs at /home/yellowtent/platformdata/logs/updater/cloudron-updater-2020-09-12_07-34-03.log Sep 12 09:34:03 box:shell update (stdout): Running as unit: cloudron-updater.service Sep 12 09:34:03 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 09:34:08 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 09:34:13 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 09:34:18 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 09:34:23 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 09:34:28 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
-
@girish it was another error message but forgot to take a screenshot. It was something like “Cloudron couldn’t start” with a link to the log.
-
imc67 translatorreplied to girish on Sep 12, 2020, 9:47 PM last edited by imc67 Sep 12, 2020, 9:50 PM
@girish my Cloudron #3 also got the same error, this time with screenshot and log:
Log:
Sep 12 23:44:27 box:tasks 2022: {"percent":70,"message":"Installing update"} Sep 12 23:44:27 box:shell update spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/update.sh /tmp/box-1847933296 Sep 12 23:44:27 box:shell update (stdout): Updating Cloudron with /tmp/box-1847933296 Sep 12 23:44:27 box:shell update (stdout): => reset service cloudron-updater status in case it failed Sep 12 23:44:27 box:shell update (stdout): Failed to reset failed state of unit cloudron-updater.service: Unit cloudron-updater.service is not loaded. Sep 12 23:44:27 box:shell update (stdout): => Run installer.sh as cloudron-updater. Sep 12 23:44:27 box:shell update (stdout): => starting service (ubuntu 18.04) cloudron-updater. see logs at /home/yellowtent/platformdata/logs/updater/cloudron-updater-2020-09-12_21-44-27.log Sep 12 23:44:27 box:shell update (stdout): Running as unit: cloudron-updater.service Sep 12 23:44:27 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 23:44:32 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 23:44:37 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 23:44:42 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 23:44:47 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds Sep 12 23:44:52 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
-
@girish I've sent you the log file via email
-
There is a SOGo regression in 5.6 . https://forum.cloudron.io/topic/3161/sogo-connectivity-issue-with-cloudron-5-6 has the bug report and workaround.