Automated env configuration destroys InvoiceNinja custom mail configuration on every restart

[foliovision]

@necrevistonnezr said in Automated env configuration destroys InvoiceNinja custom mail configuration on every restart:

The uuid is unique, don‘t copy and paste the commands

That's not our uuid. 230be9ee-72d6-4c54-8218-b08f8d217666 is a random character string. And I really don't need to talk more about NextCloud on this thread. This thread is about InvoiceNinja and how Cloudron overwrites our custom settings with no notification.

[nebulon]

To take a step back here, the reasoning for resetting email configs for most apps, is that we take an approach where apps will ensure their Cloudron related settings on each restart to make sure the app is in a known state. The start.sh can be rerun and will always attempt to bring it to what we expect. This is to ensure a configuration in case a user has accidentally changed important configs, not just email but also database connection details, file permissions and the domain the app is running on. We mostly follow the same principle as ansible playbooks and the likes here. This is the only realistic way to deliver tested updates and be able to debug things efficiently.

Email configs though is not the same for all apps, as there are mainly two broad categories. One is transactional emails and the other is mass email like a newsletter. Those two require different strategies usually.

I guess the main issue with Invoice Ninja here arises, since we think InvoiceNinja falls into the transactional category, where deliverability is ensured via the Cloudron platform setting level. Which is why it gets overwritten on every restart. For newsletter apps, ideally they have two email configs for that matter, where the transactional will also get overwritten by our startup scripts and the mass mail sending goes via some relay suitable for this use-case.

So if you hit deliverability issues within InvoiceNinja, you likely also hit this in other apps. So having Cloudron wide email settings solved for this will also help with other apps.

Question now is why you have the need for having a custom setting for this one app, which in my view does not send out mass email?

From the UI perspective, I do agree, that this is not great that InvoiceNinja even allows you to configure email settings inside the app. In an ideal world, we would probably hide that UI to avoid all this, but we are not the developers of those apps and contributing upstream is currently often unfortunately not possible as we don't have the capacity and time for this. Hiring more people who can work on that aspect, would be great, but this is currently, also given the price-point, not viable.

[jdaviescoates]

My 2p:

• Cloudron is awesome.
• Their decision actually makes lots of sense for the vast majority of their users, in large part because...
• "make sure the app is in a known state" that @nebulon mentions is what makes backing up, cloning and migrating apps all remarkably easy and reliable.
• Backups actually work brilliantly most of the time. When they don't it's normally nothing to do with Cloudron, but instead to do with network issues or quirks of where one is backing up to.
• Nextcloud is evidently viable given it's by far the mostly widely used solution of its type and is used by millions of people from individuals up to large Gov'ts.
• If this has been a terrible pain point for 2 years, perhaps it should've been reported it 2 years ago.
• Being obnoxious is generally not a useful nor productive strategy for resolving issues.

That is all.

[foliovision]

@nebulon said in Automated env configuration destroys InvoiceNinja custom mail configuration on every restart:

Question now is why you have the need for having a custom setting for this one app, which in my view does not send out mass email?

I've answered that question above. Deliverability. I don't care about deliverability on the other apps, as they go to internal addresses and internal servers which are set up to accept Cloudron email (we have control). But that's not the essence of the question. The essence is Cloudron management sabotaging its users.

Stop tampering with our email settings @nebulon please. Your high-level view of how we should run our email is annoying and totally out of place. If the preferences are there don't overwrite them.

We do not want our email overwritten. On spinup, feel free to put some defaults in. On reinstall (if an app has been erased), feel free to put the defaults back in of course.

If I have changed settings within an app, do not touch them please.

I'd like to see change here, as the structure as it is now is simply wrong-headed.

I know you don't want to ruin your users lives @nebulon. But that is what you are doing by overwriting key settings without explicit warnings.

[foliovision]

@jdaviescoates You are welcome to your 2¢. I disagree with you and have enumerated exactly why. I have not said NextCloud is not viable. I've said running NextCloud is a huge waste of development/admin resources without a userbase of at least 20.

[foliovision]

@robi said in Automated env configuration destroys InvoiceNinja custom mail configuration on every restart:

@foliovision is there a setting for nextcloud to not save trash data? Or only keep it for a short time?

Yes, but it looks like Cloudron defaults to keeping data forever, requiring those of us running NextCloud to dig very deep to figure out how to get rid of deleted files. Intelligent defaults here would be useful. NextCloud is such a bear that I don't blame Cloudron admins for throwing up their arms and telling users, "You want to run NextCloud? Learn how to admin it properly." InvoiceNinja is in another category. It's relatively easy to setup and easy to use.

Yes, users should not be changing database settings, that's too tied into the automation as @girish suggested. Totally understand. Email configuration can be set to default on first launch but then should not be tinkered with if the user changes email settings, particularly for apps where email is core to their functionality.

@nebulon already recognised this issue, which is why there are custom settings for email programs. nebulon should realise that email deliverability has changed in a cardinal way since he set up Cloudron. Email has gone from most email gets through unless there was a previous major spam issue from a given account to email does not get through unless the major ISP has been paid off (partnership relationships) or there is a full SPF/DKIM/DMARC setup, which is not entirely possible with Cloudron from what I can remember (we have setup what we can) AND you are not hosted on DigitalOcean (the recommended provider of Cloudron).

[necrevistonnezr]

@jdaviescoates great points.
Some quite rude know-it-betters want a turnkey solution without admin hours (as if OneDrive or Dropbox would not require admin hours in a large organization - at a totally different price point) and I guess save a few bucks by not using a reliable mail relay system-wide.
Maybe the it‘s just not the right product?
Anyway, engaging is just exhausting and blocking in this forum is there for a reason

[robi]

How about a feature request that adds a user configurable (env?) file for non-default settings?

Should satisfy both needs.

[foliovision]

@necrevistonnezr said in Automated env configuration destroys InvoiceNinja custom mail configuration on every restart:

OneDrive or Dropbox would not require admin hours in a large organization - at a totally different price point

Dropbox requires almost no admin hours. If Condoleeza Rice had not been appointed to the Dropbox board in something like 2012 and if the Patriot Act had been repealed, Dropbox would definitely be the far better solution for filesharing between multiple computers. It's cheaper, easier and more reliable. NextCloud is fairly easily hacked (or backdoored through DigitalOcean) but it's not a systematic hack which gives the NSA and CIA direct access to all one's files, as if they are on Google Drive or Facebook or IMAP email.

[foliovision]

@nebulon said in Automated env configuration destroys InvoiceNinja custom mail configuration on every restart:

In an ideal world, we would probably hide that UI to avoid all this, but we are not the developers of those apps and contributing upstream

This is a backwards approach. It would be completely in Cloudron's control to check the .env variables for every app. If those variables change, Cloudron should:

1. notify the admin user with a screen on first startup alerting the user that key variables have changed (which ones)
2. send an email to the admin user alerting the user that key variables have been changed

I don't think local admins should have control over database settings but we should certainly be alerted if they are being changed back and forth.

It would also be very simple to allow users to enable or disable centralised email rewritten in each app. It's just one checkmark in the admin interface for the particular install.

That you have not spent the time to think about these issues and/or fix them is deeply disappointing. Your insouciance about how awful the situation is now is a deep indictment of the open source model (and this is from someone who has spent most of his adult life both creating open source and funding it).

Cloudron is sinking under too many apps and not enough care and housekeeping around the key ones you do support. I know the too-much-to-support issue first-hand. Due to WordPress making it ever more painful to keep our FOSS plugins both up to date and active in the directory (there were code change barriers, but even more administrative barriers), we had to retire about ten of them. Users can't find them or use them.

@girish does a great job running around to help us all and give us tips. It's outstanding. Systemic issues like this need to be addressed in a deeper way and not waved away with one blithe hand, or swept under the carpet.

Don't let the yes-men flatter you into complaisance. Unmerited pats-on-the-back is not how one improves a system or makes progress.