Hey there,

I'm not a paying customer (yet) so don't feel any obligation or pressure to do something like this... I'm not sure if your customers actually need something like this actually. Although I'd find this useful myself.

Currently, for backup retention, there is a dropdown of different types of retention schedules. These retention schedules are quite varied, and the ones towards the bottom are more geared towards the rsync backup option, I believe.

I like the options further down in the list (like 2 daily, 4 weekly) because they provide options if someone messes something up and you don't notice it for a while. However, currently I've been sticking with the tarball option because it's just a lot faster for backing up things like NextCloud. The downside though, is that tarballs are a lot larger, and running a 2 daily 4 weekly tarball backup uses up a lot of space.

I was wondering if it was at all possible for Cloudron to offer a few more retention policy options... things like '2 daily 2 weekly' (although I'm not necessarily sure this is the particular one you guys want). Whether that's by adding more options in the dropdown, or allowing us to set a specific number of daily, weekly, and monthly backups retained - I think either would be pretty awesome.

In any case, let me know what you think, and thank you for all of the work that you do. Your software is really great, and thank you for all of the hard work you guys do making this software which makes installing and backing up these apps way way easier.

Just wanted to give everyone an update.

Last week was pretty crazy for me, so I wasn't able to do much testing.

I will probably resume doing some more tests this week. In any case, restarting dovecot every hour does seem to be a suitable mitigation for anybody who has this issue (at least pending until a solution is found).

My use case is a little different than yours, so I don't have a tutorial, but Directus shouldn't be terribly hard to integrate with a JAM website.

I'm currently using it as a source for a static website using Elder.js. Elder.js isn't a JAM - it's a static site generator - but it works kinda similarly. Directus works wonderfully for it.

In most of the JAM stacks, there is a point for fetching data, and at that point (assuming your using Node.js here), you just use axios to call the endpoints that Directus exposes. When you want to create user accounts or whatnot, you make an API call to Directus.

I don't know how well Directus would stand up to a busy traffic website performance wise, but I'm pretty sure it would do a great job for a low-to-moderate traffic website. It might be a good thing to ask them on their Discord, they are very friendly.

A good way to picture Directus is that it's a nice frontend to an SQL database. So a lot of the things you'd care about with a regular database will matter with Directus. It's not going to automatically manage the accounts for you - but it's not hard to create a collection called 'Website Users' and just add a user account when someone registers on your site, and then verify it when they log in, managing the cookie or token appropriately.

This of course means it isn't 'low-code', but it isn't 'that's a ton of code' either. (As a note, if your going for posts and logins and the likes, I'd stick with JAM, and stay away from static site generators - static site generators are really built for sites that don't have live content, not that you can't fenagle your way around it, but it's not what they are built for).

If your looking for a bit more automation then that for logins... there's a few other frameworks that have built in libraries for user authentication, but the only one I remember really standing out as being 'easy' was Meteor (using their old UI framework called 'Blaze'). But Meteor is pretty different from Directus, and it's very opinionated, and usually requires a heavier server. Might not be what you are looking for.

One thing I will warn you on Directus... sometimes, major version upgrades (think 8.?.? -> 9.0.0) don't work quite as planned (at least in my experience). So if you see they are going to put out a new major version soon, it might be worth holding out until they do. Just my personal experience though.

Best of luck with your project!

Hey Everyone,

I was just wondering what VPS's you all use for hosting your Cloudrons? I figured it might be helpful for those who are looking to get started in this.

Currently I use BuyVM for my VPS, and use their very cheap Block Storage ($5/TB/month!) for a ton of cached hard drive space. It works really well actually.

For backups, I am currently trying that with a Hetzner Storage Box currently using a NexusBytes box for backup (much closer to the VPS than Hetzner was, which means speeds are about 20x faster). Would have used BuyVM for this as well except that I like to host the VPS and Backup on different services.

What does everyone else use?

@girish This makes a ton of sense, although I'm not quite sure if I'm doing it right. Under what you linked to, the section should be under the Resources tab, although I think it's moved to the Storage tab (no problems with that, I just wanted to make sure I'm handling it correctly).

The part that's a little confusing is how 'App Data' and Volumes tie together in storage (now it seems this may be resolved as you said, since your working on Volume updating at some point in the future).

Anyways, App Data says to mount an external EXT4 disk, which I assume is done with the 'Mounts' interface right below it. However, the mount says that you are mounting 'Volumes'.

The documentation says 'App Data' is backed up. However, the documentation also says that 'Volumes' are not backed up. So that does leave me a little perplexed as to whether the data will be backed up or not. I think according to your comment, that it will be backed up (because you did say it would no matter where it is on the server). But I just thought I'd check

In any case, thanks for your help, and hope you have an awesome rest of your week.

@humptydumpty Hey, so I have some Feedback on the Hetzner Storage Box. If you use it in Europe, it'll probably be all right, as you are a lot closer to the box.

However, for US Backup Storage of any sizable amount, it's going to be very slow. I was looking at around 60 days or so to upload 300GB (probably mostly because of the distance). So I'm looking for something a bit closer.

My ping response time for Hetzner was 146 ms, which isn't terrible... but I can get something around 7 ms with a VPS closer to home, which I think will work a lot better for the backup. We will see though.

@d19dotca I can't believe I missed that. Literally read right past it. Thank you!

@BrutalBirdie Hey Brutal, actually if it's cool, let me do a few tests first. I think I might have isolated a particular issue... and it might be unique to thunderbird... if it's just a thunderbird issue, it might not be related to the email server at all.

I'll let you guys know later today... unfortunately, today is likely to be crazy busy... just did a really large email migration, and those are kinda painful sometimes :).

Hey all,

I was hoping some of you might have a bit of guidance on some email issues I've been having. If not, that's cool too though :).

I'm running about 300G of email on a Cloudron System. Currently, right now, I'm running into a case where sporadically, some IMAP and POP folder reads will time out. These aren't huge folders either - some only have 2-3 files. SMTP is not affected.

There's some extraneous situations which might be related to this. Two slightly problematic things happened over the past few weeks. (1) I ran an external backup disk out of storage - it isn't tied to the disk the mail is on in any way. (2) I had to Hard-stop the server on the VPS provider side of things because of a configuration issue.

I'm not entirely sure which of the two instances caused the issue, but it has been occurring ever since. I'm wondering if some files got corrupted or something.

One other thing I've noticed while doing this is that the mail docker instance is taking up a lot more memory. I've raised it from 500K to 1G to 2G, and sometimes it still even fills up that. I'm not sure if that's related or not - but the issue does occur whether it is full or not - I've had cases where the issue occurs when only using about 1G memory out of 2G allocated to it. It does tend to happen a bit more when it's full though.

When I've run a Cloudron Email Server in the past, it's not really had this issue of memory filling up, but they've also been for far less email (10G or so). Then again, those have all been sending email directly, not through SES.

Restarting the mail process fixes the issue until it happens again. However, it's very difficult to predict when the issue will happen again.

I've gone into the mail image inside and edited the dovecot config and turned on mail_deubg and auth_debug - I'll post some information from the logs when I get some information from this. EDIT: Not seen anything significant in logs. Will probably wait for another occurrence and see if I can spot any patterns.

I'm trying not to rely on the Cloudron Support team because I have not bought Premium yet (will probably do in the future, but I'm not quite ready to do so at the moment).

Have any of you seen issues like this before? If so, how have you proceeded towards fixing it?

Another update. Updated the 2007 Outlook Client to Office 365. Haven't had any issues with them so far, but need to wait to be sure. There's a small chance that this whole thing was actually two separate email client issues, one with Thunderbird, and one with Outlook.

Certain versions of Outlook actually have an issue where they don't respond to Dovecot's request to see if they are 'still there' and if an IDLE IMAP connection is closed, they will go offline automatically (https://wiki.dovecot.org/Timeouts and also see point 2 and 3 of the last post in https://social.technet.microsoft.com/Forums/exchange/en-US/b418d42a-ce66-4cfa-adb9-e6e24ec769f4/outlook-2007-error-your-imap-server-closed-the-connection). I'm thinking this might have been the issue. Not seeing this issue on Office 365 clients other users are running.

As for the Thunderbird issue... found one account of someone running into something similar here: https://www.mail-archive.com/dovecot%40dovecot.org/msg82805.html . Still gotta work on that one, and do some measurements to see if we are hitting the process limit when we have the issue. This would be a weird case though as we are only seeing this issue with some Thunderbird clients, despite all running the same version and with similar software on the computers... which again points to a mailbox level issue. Gotta do some more sleuthing.

@Azzer Which of Cloudron's DNS Settings are you using?

Are you using Manual Setup (where you manually edit the DNS settings), or are you using Automatic Setup (where you provide Cloudron a Secret Key to change your DNS for you).

@makemrproper Hey makemrproper,

If you want a VPS with a lot of space, you might try looking into BuyVMs storage volumes.

They take a bit of time to set up (you have to mount the volume), but it's very difficult to get a deal on the space provided elsewhere.

It's also very well cached, so I've found it to be almost as performant disk wise as what I've used on digital ocean.

They are a smaller provider though, so the reliability won't be quite as good (you might have a little more down-time compare to other providers).

I know this doesn't necessarily solve your issue, but more disk space is always great :).

Just wanted to give everyone an update.

Last week was pretty crazy for me, so I wasn't able to do much testing.

I will probably resume doing some more tests this week. In any case, restarting dovecot every hour does seem to be a suitable mitigation for anybody who has this issue (at least pending until a solution is found).

Another update. Updated the 2007 Outlook Client to Office 365. Haven't had any issues with them so far, but need to wait to be sure. There's a small chance that this whole thing was actually two separate email client issues, one with Thunderbird, and one with Outlook.

Certain versions of Outlook actually have an issue where they don't respond to Dovecot's request to see if they are 'still there' and if an IDLE IMAP connection is closed, they will go offline automatically (https://wiki.dovecot.org/Timeouts and also see point 2 and 3 of the last post in https://social.technet.microsoft.com/Forums/exchange/en-US/b418d42a-ce66-4cfa-adb9-e6e24ec769f4/outlook-2007-error-your-imap-server-closed-the-connection). I'm thinking this might have been the issue. Not seeing this issue on Office 365 clients other users are running.

As for the Thunderbird issue... found one account of someone running into something similar here: https://www.mail-archive.com/dovecot%40dovecot.org/msg82805.html . Still gotta work on that one, and do some measurements to see if we are hitting the process limit when we have the issue. This would be a weird case though as we are only seeing this issue with some Thunderbird clients, despite all running the same version and with similar software on the computers... which again points to a mailbox level issue. Gotta do some more sleuthing.

An update on those possible issues.

Port 993 over SSL worked in SOGO, even when a desktop client was having issues.

This means that it is likely an issue with (3). I'm going to see if I can change dovecot.conf so dovecot kills long lasting connections quicker and see if that makes the problem go away or exacerbate it next.

@plusone-nick Hey Plusone, sorry if I'm just jumping in, have you thought about setting the password to a random value server side and then sending them a change password link? Not sure if that would help much at all though.

Hope you are having an awesome day.

Some updates:

The issues is still happening despite the server reboot.

I was looking into disk issues, but I'm not so sure of that anymore.

Some things to note:

Sogo Webmail is still working when the issue occurs. This leads me to conclude a few things:

• It means the issue is probably in 1 of 4 places:

1. It could be an issue in how port 9393 is forwarded over to 993 (or at least I'm assuming there is port forwarding from 9393 to 993 - I can check if this is the source the next time the issue occurs using Sogo to connect to both ports)
2. It could be an issue in how the mail is tied to the domain somehow. Sogo attempts to connect to the domain mail and it works fine. IMAP connects to my.insertdomainnamehere.com. (Once again, I can check this with Sogo the next time there is an issue).
3. There could be an issue in how dovecot is handling long lasting TCP connections (think the ones that go to IDLE). (don't know how I'd test this though tbh)
4. I suppose there could still be an issue in the underlying storage (an RDMA) that is causing issues - though I don't see why it wouldn't list any errors in the logs at that point.

• It does mean that it's probably not an auth issue as Sogo is logging in fine

In the main time... probably gonna cronjob supervisorctl restart dovecot every few minutes. Won't be perfect but it restarts within a second, and is a lot quicker than restarting the entire mail process, so it probably won't be noticeable for end users.

@girish I was wondering... perhaps the 'mail' docker image is having an issue (I doubt this, but it's I guess a possibility). Is there a way to re-download the docker mail container for Cloudron without causing any issues? (should I just docker image pull it, or would that break things?)

Update.

Had some more issues early today even after the export, deletion, re-creation, and import of the mailboxes.

However... due to some circumstances, I had to power off the VPS. When I rebooted it, I reduced the mail process down to 1GB of memory (raising it to 3GB had caused the crash)... and have not had any issues since then. Crossing fingers it stays that way.

top is showing about 50% memory use, so only a little swap, which is fine. Hopefully I can get to no swap use in the future though.

However, the real test will be seeing what happens on Monday and Tuesday. If it's good on those days, I will be very happy :).

@girish Thunderbird, Outlook 2007, Outlook 365. Only receiving the issues on Outlook 2007 and Thunderbird, and only on some clients, so it seems to be tied to certain accounts.

I'm trying a few things to see if there are corrupt mails in inboxes though... this might be tied to that instead. I'll let you know.

Update: No errors, even in the full dovecot logs with mail_debug turned on. However, I am noticing that this issue is occurring often when people log into a certain email account. As a result... I'm going to try and imapsync the data over to another account and imapsync it back... and see if this resolves the issue. May also try different email clients... perhaps this is only caused by certain email clients connecting.