Solr and Adding a New Mail Domain
-
Two questions:
(1) So I enabled Solr on one of my Cloudrons and I was curious if there is any way to see if the initial indexing has completed? The Full Text Search field shows a status of Enabled/Running. After enabling Solr, I can see a small, "permanent" increase in RAM usage (this makes sense) and the email searching from the client side is very responsive. The mail service and SoGo have limits of 4GB with more RAM on the server, so there seems to be plenty of headspace for Solr. How can I know when the process is complete or run a command to verify?
(2) So on another server, I periodically need to add another mail domain. Domains could be added either with a supported DNS Provider or in other cases I need to select Manual. After adding the domain, it looks like the Let's Encrypt certificate needs to be regenerated to include the new domain. When that happens, my iOS users on other domains experience an invalid password window, prompting the user to enter the proper password or cancel. iOS is not discriminating on these errors, so even a networking error between the user and the server or an invalid SSL certificate triggers this unhelpful message. Besides the facat that it keeps popping up relentlessly, it often encourages users to enter a password thinking this will fix the problem (it won't) and they enter the wrong password by mistake - breaking their email installation. Is there any way in Cloudron to keep the old certificate in place until a new one can replace it? If there is another way to prevent current email users from receiving that ridiculous iOS message I am open to any idea.
Thanks. Any advice on either is appreciated...
Brad
-
@crazybrad said in Solr and Adding a New Mail Domain:
How can I know when the process is complete or run a command to verify?
If the search works in the Client, the indexing is already done. From what I have seen, Dovecot doesn't reply when it is still indexing a mailbox.
-
@crazybrad for 2.. I think what is happening is that a mail server restart is confusing iOS. It's most likely not to do with certs since certs of old domains are not changing. Can you check if you can reproduce the behavior with just a mail server restart ? (Services -> Mail -> Restart) . We restart the mail server when the list of domains it needs to manage is updated.
-
@girish I tried restarting the mail server twice. I did not see the strange iOS messages either time. In checking with 2 other users, they did not see the messages either. It could be also that SoGo is "sleeping" (this seems part of the EAS mechanism) at the moment I restarted the server. If the clients were told "don't bother me for 30 seconds" then perhaps they are not requesting anything during the timeout period.
BTW, this is not just a Cloudron/SoGo issue. I have seen this behavior on iOS whenever there is some connection issue between the device and the mail server. In some cases switching from cell to WiFi or vice versa can eliminate the issue. But if the server is at fault, nothing helps (obviously).
-
-
@crazybrad I forgot about SOGo in the setup . So another thing to test: does restarting SOGo cause these errors in iOS apps? When the mail service is restart (which is done when domain is added), the email apps (like roundcube, sogo, snappy) are also restarted. This is done to make them aware of the newly added domains. Ideally, they should be able to get the domain list via LDAP but support for this is in inconsistent.
Possibly related to this: I have a simialr issue of what you describe but with thunderbird periodically requesting my sogo password for accessing the calendar. Re-supplying the password or restarting SOGo fixes this. In fact, this has been a fairly recent development (last 2-3 months). You might be right this might be EAS related since that change was added to SOGo 2-3 months ago. Let me try disabling ActiveSync in my instance and get back.
-
@girish Now that you mention the Thunderbird calendar password issue, I experienced that once as well. I assume that in the migration to Cloudron/SoGo I had missed something, but seems not in this case. I am thinking there will be more of these messages in my future...
Also, I was also shocked about how resource intensive EAS is. When I realized how many workers were "needed" from viewing the Sogo log and doing some simple math: (# users) x (# mailboxes/user) x (# devices) with those mail accounts, I think I wound up with 150 workers to achieve the right balance of responsiveness. Fortunately I am running this on a server with big CPU and RAM. 1 user with 8 mailboxes on 3 devices (desktop, tablet, phone) could trigger 24 simultaneously requests! @fbartels Any advice based on your Kopano knowledge with EAS?
-
@girish said in Solr and Adding a New Mail Domain:
When the mail service is restart [..] the email apps [..] are also restarted
Without having explicitly tested this, but I think there is a possibility for clients getting confused when clients briefly see:
This app is currently not responding. Please try refreshing the page in a few minutes.
While still a http return of 200 is given (which does seem to be the case here). But on the other hand I also don't think that this specifically could trigger password prompt, since whatever the client sends as basic auth here would just be accepted by the server and not rejected.
@crazybrad said in Solr and Adding a New Mail Domain:
Any advice based on your Kopano knowledge with EAS?
I'm not overly familiar with Sogo, but you mentioned desktop so I am guessing you are referring to Microsoft Outlook on the desktop. ActiveSync was never really made for desktop applications and therefore potentially syncing gigabytes from many folders (coupled with very large request/response sizes) can put quite some load on the server and in my experience also leads to issues on the client sooner or later. That is why we have abandoned that approach and switched to an implementation of mapi over http for this client in our latest product. The sogo documentation already lists some of the limitations of that approach as well.
On point that is made here is problematic from an implementation standpoint:
ActiveSync clients keep connections open for a while. Each connection will grab a hold on a sogod process so you will need a lot of processes to handle many clients. Make sure you tune your SOGo server when having lots of ActiveSync clients.
Blocking a worker for an idling open request will waste a lot of resources. We had the same issue in the past, but for us it was just a php connection being held open. For better scalability we are now parking these connections in a separate "proxy" to free resources in our actual server. But this is something that Sogo would need to implement (as in develop such a proxy and hand the connections off) and nothing that you as the user can influence. Generally speaking as long as you have set your client to use "push" there is nothing you can do to lessen the load on the server.
-
@fbartels Much appreciated. After your explanation (and watching what is going on with SoGo workers and how many workers are needed), my next Cloudron will not use SoGo EAS simply because it consumes too many resources. For a limited set of users (< 30), I can afford the extra resources. But if I was providing email for 200+ users, I think Sogo EAS would crash and burn.
I also hadn't thought about the consequences of having 5 email accounts on 3 different devices. While all of them may not be "active" at the same time, the load is much greater than the number of email accounts.
