What's coming in 5.6

[girish]

@Mallewax Yes, I have added it in our 6.0 list now - https://forum.cloudron.io/topic/2760/what-s-coming-in-6-0 . In fact, I just moved to a new device and search not working reasonably is a bummer.

[marcusquinn]

Nice work on the Mail server side of things!

Being able to set mail server subdomains to the more commonly used mail. will be a bonus.

Thunderbird checks common subdomains as a second resort, although the extra DNS records for the standard autoconfig stuff would be better for more clients to autodetect.

I'm hoping that will help with email client auto-setups but anything you can do to also add the necessary records would be super handy:

• https://forum.cloudron.io/topic/1816/email-autoconfig-for-autoconfiguration-example-via-thunderbird/5
• https://forum.cloudron.io/topic/1296/add-dns-dav-registring/4

[girish]

Yeah, I would love to get the autoconfig functional in this release. I haven't gotten to that yet, and will probably need help of everyone to test auto-detection in variety of clients.

[Mallewax]

@girish Love it....:-) Thank you for your consideration!

[Mallewax]

@girish Maybe this is of help, too....

https://workaround.org/ispmail/buster/mail-client-auto-configuration/

[girish]

We now have server side signatures as well. These can be set per-domain:

I have also made some good progress with NC and WP optimizations. I will make a separate post for that.

[girish]

@doodlemania2 It turns out collecting network information for each container is quite complicated. It's easy to collect the stats at a point in time (like docker stats does) but to do it over a time period, requires us to periodically track the container/process. I put some notes here - https://git.cloudron.io/cloudron/box/-/issues/734 . I have to postpone that task since it's quite complicated. If someone has better technical knowledge on this, please do leave a note on gitlab.

[doodlemania2]

Thanks for the update good sir!

[d19dotca]

Any ETA on 5.6? Can't wait for these mail improvements.

[girish]

@d19dotca It's getting here, a couple of things left. Hopefully late next week

[girish]

For the email auto-configuration, for now I will just add docs on how to set this up manually. It's a bit complicated to automatically set all this up because outlooks requires SRV records (for which we have to write automation code for the DNS providers) and thunderbird/k9 requires setup of .well-known files. This has the same issue as mastodon where we can automate only the website is also on Cloudron. So, for now, I will put docs on how anyone can set this up manually.

[will]

@girish Can Cloudron not write to the "well-known" folder on the host?

I was thinking matrix could be a switch in the admin panel.

[girish]

@will It can but it only works when the domain's web server is hosted on Cloudron. For example, cloudron.io website is hosted on a different server than cloudron.io email. The well-known has to be added on whichever server is hosting the website and not the email server.

[will]

@girish I would guess that most people have everything they can on cloudron, is that the reality in the field?

[marcusquinn]

@will I see what @girish is saying:

• One Cloudron managed server has Email hosted with it's various subdomains
• Another (maybe Cloudron managed) server has the root website domain.
• So the .well-known file needs hosting on the root domain server.

I've been thinking a fair amount recently about whether to www. the main website or not.

Originally, I though, who still types www.?

But, that misses the ability to manage it as just another subdomain, and have the main root domain 301 redirect to www..

Reasons for having www.? Well, this use-case could be one. CNAME load-balancing could be another.

I'm thinking that Google does www. for their main domain and many other big players do too, so maybe they are right and my non-www. thinking for succinct brevity was missing some of these other possibilities that using www. enables.

Might be an SEO risk changing large websites and potentially messing up the 301 redirects for all indexed URLs but I'm having a serious think about going back to www. given those using it probably know more than I do as to why it might be better.

[d19dotca]

@marcusquinn For most people it doesn't matter if we use www or non-www, but I found this page many years ago which has made me stay consistent in using www over non-www, which you may find interesting too.

It mostly comes down to technicalities really (but this is probably a good example of one too): https://www.yes-www.org/

Or more specifically, this direct page for the reasons why to use www over non-www: https://www.yes-www.org/why-use-www/

With that said though, while the vast majority of major websites use www, some very large companies don't either, so there must be some valid reason or workaround they've found to any restrictions. Twitter is one that comes to mind, for example. That article actually mentions Twitter too. haha.

[marcusquinn]

@d19dotca Very handy, thank you - you and they have me convinced now.

We have a load of Kubernetes stacks - but I feel they are overkill for most of your needs and the pods and replication has network latency, so I'm looking at more of a master-slave hot-swap high-availability strategy that will ultimately be cheaper, easier to understand and simpler to scale.

[girish]

Another progress update. There is now a very basic firewall configuration UI to block IPs. You can use this to block connections to the mail server as well. In future releases, we will probably make this more dynamic. For example, just automatically block IPs for a specific period of time, if there are too many failed logged in attempts. We have to implement this across apps and the app logs are not too fail2ban friendly right now, so we have to see how we can do this.

There is also a mechanism to white list ports (will be documented) but there won't be a UI for this since I am not sure we should go about encouraging people to install things on Cloudron.

I think apart from some UI adjustments that @nebulon is making, we are set for the release!

[d19dotca]

@girish Just wanted to check for an update on this as I am really needing to store emails on a different location just like how apps work (my medical clinic clients are using email massively compared to before because of COVID-19 testing they've been doing the last few months, and the storage space usage is growing very quickly. I know you postponed the mail backup part where it was separated from box backups, but does this negatively impact the ability to store emails on an external disk too or is that feature still coming for storing emails on external disks?

[girish]

@d19dotca said in What's coming in 5.6:

I know you postponed the mail backup part where it was separated from box backups, but does this negatively impact the ability to store emails on an external disk too or is that feature still coming for storing emails on external disks?

Yes, we postponed the splitting up of box data and mail data. But moving mail to a separate disk is already possible. Just that box data also goes with it. This should not be a problem for your situation. Essentially, the directory /home/yellowtent/boxdata (which contains the emails) can be moved anywhere you want. See https://cloudron.io/documentation/storage/#default-data-directory . Essentially, stop docker and cloudron, copy over all the stuff to new location, then just create a symlink to the new location.

(Just make sure you have backups, just in case). Note that the external disk has to be ext4.

[d19dotca]

@girish Dang, that's too bad. Thank you for confirming though. I'll keep that process in mind as something I can do when the time comes then (which I suspect will be in about 2 months or earlier), but hoping there'll be an easier way to still have the box data on the local disk rather than needing to move it in its entirety to a new disk. Just as how the app locations can be changed and it automatically moves the data, it'd be great if Cloudron could do that part too without needing a manual intervention for it. Plus it'd save a little bit of cost to the admin as they wouldn't need to create such a large external disk for it all, just for the emails.

[girish]

The 5.6 pre-release is out. Only update if you are truly eager. We will roll this update out very slowly over this week and the next.

If you hit any issues, please do open separate threads. The docs are still getting fixed up, so let me know if you find something hard to follow.

[mehdi]

@girish said in What's coming in 5.6:

• Add a way to whitelist inbound ports. This will help apps like netdata, snmp monitors and other monitoring "agents".
• Add flag in manifest for reverse proxying to a port (for the OLS app).

Could I have more detail about these 2 features ? They look quite interesting !

[girish]

@mehdi said in What's coming in 5.6:

Add a way to whitelist inbound ports. This will help apps like netdata, snmp monitors and other monitoring "agents".

https://docs.cloudron.io/networking/#whitelist-ports

As for the port proxying, it didn't make it to this release. It was meant for the OLS app that @MooCloud_Matt packaged. We have to try it for the next release.

[d19dotca]

@girish I just updated to 5.6.0 to test it out. Looks great so far! Only thing I'd suggest changing or fixing is the Max Message Size on the Email page, as it's default at 25 MB but when I go to change it, it goes in weird 10 MB intervals from 1 MB, so it's 21 MB, 31 MB, 41 MB, etc which are strange numbers to use. I'd suggest it either be configurable (i.e. a text box) or if it's on a slider then it should go up in 1 MB increments rather than 10 + 1.

[girish]

@d19dotca Ah thanks, I can reproduce this easily. Strange

Edit: I have fixed this for next release.

[imc67]

Update to 5.6 failed with error but seems to work anyway.

Log:

Sep 12 09:34:03 box:tasks 4283: {"percent":70,"message":"Installing update"}
Sep 12 09:34:03 box:shell update spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/update.sh /tmp/box-2509650539
Sep 12 09:34:03 box:shell update (stdout): Updating Cloudron with /tmp/box-2509650539
Sep 12 09:34:03 box:shell update (stdout): => reset service cloudron-updater status in case it failed
Sep 12 09:34:03 box:shell update (stdout): Failed to reset failed state of unit cloudron-updater.service: Unit cloudron-updater.service is not loaded.
Sep 12 09:34:03 box:shell update (stdout): => Run installer.sh as cloudron-updater.
Sep 12 09:34:03 box:shell update (stdout): => starting service (ubuntu 18.04) cloudron-updater. see logs at /home/yellowtent/platformdata/logs/updater/cloudron-updater-2020-09-12_07-34-03.log
Sep 12 09:34:03 box:shell update (stdout): Running as unit: cloudron-updater.service
Sep 12 09:34:03 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 09:34:08 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 09:34:13 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 09:34:18 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 09:34:23 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 09:34:28 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds

[girish]

@imc67 are you referring to "Failed to reset failed state"? That message can be ignored.

[imc67]

@girish it was another error message but forgot to take a screenshot. It was something like “Cloudron couldn’t start” with a link to the log.

[imc67]

@girish my Cloudron #3 also got the same error, this time with screenshot and log:

Log:

Sep 12 23:44:27 box:tasks 2022: {"percent":70,"message":"Installing update"}
Sep 12 23:44:27 box:shell update spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/update.sh /tmp/box-1847933296
Sep 12 23:44:27 box:shell update (stdout): Updating Cloudron with /tmp/box-1847933296
Sep 12 23:44:27 box:shell update (stdout): => reset service cloudron-updater status in case it failed
Sep 12 23:44:27 box:shell update (stdout): Failed to reset failed state of unit cloudron-updater.service: Unit cloudron-updater.service is not loaded.
Sep 12 23:44:27 box:shell update (stdout): => Run installer.sh as cloudron-updater.
Sep 12 23:44:27 box:shell update (stdout): => starting service (ubuntu 18.04) cloudron-updater. see logs at /home/yellowtent/platformdata/logs/updater/cloudron-updater-2020-09-12_21-44-27.log
Sep 12 23:44:27 box:shell update (stdout): Running as unit: cloudron-updater.service
Sep 12 23:44:27 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 23:44:32 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 23:44:37 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 23:44:42 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 23:44:47 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds
Sep 12 23:44:52 box:shell update (stdout): cloudron-updater is still active. will check in 5 seconds

[girish]

@imc67 can you see what the error is in /home/yellowtent/platformdata/logs/updater/cloudron-updater-2020-09-12_21-44-27.log ?

[imc67]

@girish I've sent you the log file via email

[girish]

There is a SOGo regression in 5.6 . https://forum.cloudron.io/topic/3161/sogo-connectivity-issue-with-cloudron-5-6 has the bug report and workaround.