Cloudron Forum

Well, the thing is that I can obviously rename folders in my own account(s) in order to work around this, but when migrating accounts for other family members, colleagues or clients it's harder since I will be asking them to change their folder naming habits just because the mailsystem can't handle it.

To be honest I never ran into this with other mailsystems and I've used / tried a considerable amount of them in the meantime: Google (Gmail/workspace), Microsoft (hotmail/365), Zoho, Mailcow just to name a few and they all accept periods in folder names.
As a result we got used to use periods in our folder names.

I'm fine if the default stays as it is in order to not break anything for other people, but being able to change it when needed would be nice.
And yes, there are possible collisions with the internal folders (new, cur, tmp) when switching to FS, but this can be anticipated by setting the DIRNAME variable to something unique.

In regards with testing, I'm not sure this has a huge impact since the working of dovecot shouldn't change in any way when switching to FS.

While trying to migrate an email account from another mail system to Cloudron I noticed that it doesn't succeed because folders have a "." (period) in their name and maildir uses a "." to separate subfolders on the filesystem.

This can easily be solved by adding LAYOUT=fs to the mail_location option in the config file, but I'm not sure where I can do this in Cloudron?
Can anyone point me to the correct location to make this change?

Also, if I make this change, will it persist after upgrades?

The documentation for this setting can be found at the Dovecot website.
And for a more in depth read on the matter of periods and slashes in mail folders you should take a look at this post.

@chetbaker thanks for the input but I'm afraid this isn't anything new, cfr the posts by girish and necrevistonnezr

@girish or @nebulon , any chance this will be implemented?
This is definitely needed since marking messages as spam is not sufficient when an address isn't in use anymore.
There are even reports of hosters suspending or blocking your account when receiving floods of spam (here for example)

@necrevistonnezr that's correct, but as you might understand this was just an example.
In reality it will be something like this@mydoman.com, that@mydomain.com and whatever@mydomain.com where wildcards are not a solution.

When you want to have the same email addresses / aliases on multiple domains, e.g. the same domain with different extensions, you currently have to set up all email addresses and aliases for each domain separately.

For example:

• address1@mydomain.com
• address2@mydomain.com
• alias1@mydomain.com
• alias2@mydomain.com
• address1@mydomain.net
• address2@mydomain.net
• alias1@mydomain.net
• alias2@mydomain.net

For this you currently need to define 8 addresses / aliases.

With domain aliases however it would be possible to set up everything for mydomain.com (the first 4 in this example) and then point mydomain.net as an alias domain to mydomain.com and automagically those addresses / aliases are also available under the .net domain.
This makes it much easier to maintain in case you have many addresses / aliases.

This can be done already in other mail solutions such as Google Workspace, Zoho mail, Mailcow, etc.

Note:
I'm aware of this other topic, but the op didn't express an explicit desire to have this happen automagically and as a result that topic was marked as implemented in 5.2 with the current, manual implementation which is not what is proposed in my request.

I'm sorry I missed that post.
Maybe both can be merged, since my post provides an actual link to the API documentation?

It would be nice if ClouDNS could be added as a DNS provider for automatic DNS configuration in Cloudron.
If I'm not mistaken there are other people in this forum using ClouDNS as well.

They do offer an API for this.

Thanks!

I completely agree with this, but the thing is that even if your mails get delivered to those mail providers they might (and probably will) still end up in the junk folder of the recipient(s). Yes, I'm looking at you Microsoft.

So while I agree that a secondary / backup relay would be helpful in case of temporary delivery errors, it won't solve the problem of getting your emails in the recipients inbox in all cases.

Test case:
I did some extensive testing by sending a correctly formatted test mail from a Cloudron server that's being used for +3 years for sending out mails from several clients' addresses without any issues.
The test mail was correctly delivered to the inbox of a MS 365 account, Google Workspace account, GMail account and several other accounts, but was consistently marked as junk in hotmail, live and outlook.com accounts.
The ip address of the sending mailserver isn't on any blacklist, should have a good reputation in the meantime and mail-tester gave a 10/10 for the content of the test mail.
But even when I used a relay server (smtp2go and mailjet) the result was exactly the same.
So as a final test I decided to send the test mail from my MS 365 account and even then it was marked as spam in hotmail, live and outlook.com addresses!
In other words, they don't even trust their own mail servers.

Just as an FYI...

I noticed similar issues since the update to Cloudron 8, but in my case it was caused by the fact that the backup storage wasn't mounted correctly.
After clicking the "remount storage" button this was fixed and backups started working again.

I'm using sshfs and had this issue on 2 Cloudrons.
Maybe there is a general regression in the update procedure that causes the backup storage location to be in an unmounted state, resulting in failed backups?

@girish That's great to hear!

@girish since PHP 8.1 isn't supported anymore for almost 6 months now, aren't there any plans yet to upgrade to 8.2 or higher? Also some Nextcloud plugins require at least 8.2 in the meantime.
So both for security reasons and support reasons it's recommended to have at least v8.2.

And we're up and running again!
Thanks again @girish , without your help I wouldn't be able to get it running again.

@girish indeed, I just managed to get passed the failed migrations by making a new baseline and then telling prisma that all migrations have already run.

Unfortunately the app still won't start after the successful database migration.
It seams it tries to run another database migration during startup
Any clues?

  /app/pkg/start.sh 
==> Changing ownership
NOTICE:  extension "pgcrypto" already exists, skipping
CREATE EXTENSION
=> Running build script that generates the migrations
yarn run v1.22.19
$ npm-run-all check-env build-db check-db build-tracker build-geo build-app
$ node scripts/check-env.js
$ npm-run-all copy-db-files build-db-client
$ node scripts/copy-db-files.js
Database type detected: postgresql
Copied /app/code/db/postgresql to /app/code/prisma
$ prisma generate
Prisma schema loaded from prisma/schema.prisma

✔ Generated Prisma Client (v5.12.1) to ./node_modules/@prisma/client in 219ms

Start using Prisma Client in Node.js (See: https://pris.ly/d/client)

import { PrismaClient } from '@prisma/client'
const prisma = new PrismaClient()

or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)

import { PrismaClient } from '@prisma/client/edge'
const prisma = new PrismaClient()

See other ways of importing Prisma Client: http://pris.ly/d/importing-client

┌────────────────────────────────────────────────────────────────┐
│  Supercharge your Prisma Client with global database caching,  │
│  scalable connection pooling and real-time database events.    │
│  Explore Prisma Accelerate: https://pris.ly/cli/-accelerate    │
│  Explore Prisma Pulse: https://pris.ly/cli/-pulse              │
└────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│  Update available 5.12.1 -> 5.13.0                      │
│  Run the following to update                            │
│    yarn add --dev prisma@latest                         │
│    yarn add @prisma/client@latest                       │
└─────────────────────────────────────────────────────────┘
$ node scripts/check-db.js
✓ DATABASE_URL is defined.
✓ Database connection successful.
✓ Database version check successful.
Error: P3005

The database schema is not empty. Read more about how to baseline an existing production database: https://pris.ly/d/migrate-baseline

✗ Command failed: prisma migrate deploy
Error: P3005

The database schema is not empty. Read more about how to baseline an existing production database: https://pris.ly/d/migrate-baseline


error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: "check-db" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

@girish said in Umami not responding:

@guyds we are in sync about the situation

Haha

But to clarify, you should still be able to follow upstream instructions. It might be a bit different on Cloudron but you should atleast be able to try them out.

• You have to put the app in recovery mode. Repair -> Recovery
• Then, in a web terminal of the app, you can run whatever commands you want.

https://git.cloudron.io/cloudron/umami-app/-/blob/master/start.sh?ref_type=heads is the startup script of Cloudron. When in recovery mode, the app container is created but the start up script is not executed. So, you have to then execute the exports at https://git.cloudron.io/cloudron/umami-app/-/blob/master/start.sh?ref_type=heads#L11 manually in the web terminal. Hopefully, all this makes sense (if you have dealt with Docker before, it should be straightforward).

Thanks for the pointers regarding the exports!

Initially, after the exports, I got the same error:

  npx @umami/migrate-v1-v2@latest
Running v0.15.0
/app/code/node_modules/.prisma/client/default.js:43
    throw new Error(
          ^

Error: @prisma/client did not initialize yet. Please run "prisma generate" and try to import it again.
In case this error is unexpected for you, please report it in https://pris.ly/prisma-prisma-bug-report
    at new PrismaClient (/app/code/node_modules/.prisma/client/default.js:43:11)
    at /root/.npm/_npx/429c6c96e109f52c/node_modules/@umami/migrate-v1-v2/index.js:298:12
    at Object.<anonymous> (/root/.npm/_npx/429c6c96e109f52c/node_modules/@umami/migrate-v1-v2/index.js:323:3)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)
    at node:internal/main/run_main_module:23:47

Node.js v18.18.0

But I was able to locate the schema.prisma file, run prisma generate and then fix some other node issues until the migration finally worked:

  npx @umami/migrate-v1-v2@latest
Running v0.15.0
✓ DATABASE_URL is defined.
✓ Database connection successful.
✓ Database v1 tables ready for migration.
✓ Database v2 tables found.
✓ Database is ready for migration.
✓ Starting v2 data migration. Please do no cancel this process, it may take a while.
✓ Ran sql file /db/postgresql/data-migration-v2.sql.
✓ Data migration from V1 to V2 tables completed.
✔ Do you want to delete V1 database tables? (Y/N) … n
✓ Migration successfully completed.

Now let's try to restart the app...

@girish I understand this is not Cloudron's fault in any way.
i'm also aware that v2 is not backwards compatible with v1 in that the events can't be migrated.
And while that's definitely a real bummer, it should still be possible according to their docs to upgrade from v1 to v2 and keep all the existing accounts as well as all the websites and therefore also all the website ID's which are used by other people to look into the stats.
If I have to reinstall, I'll lose all the accounts and website id's.

Rolling back to v2.4.2 is obviously not the best solution, but even if that could temporarily solve the situation I still can't because the backups can't be restored to a running state for some reason.

But yeah, I guess I have no other option than just starting from scratch.
That'll conclude a nice end of the week after having to deal with several other incompatible software upgrades this week

Thanks anyway for you support!
And I know it's not your (nor Cloudron's) fault

@girish said in Umami not responding:

@guyds said in Umami not responding:

Isn't there any way I can try to manually run the migration script in recovery mode?

Sure, you should be able to run the migration script in recovery mode. But the issue was that the upstream migration script is not working/complete in all situations, from what I remember. But yes, give it a try .

And how can I do that?
I already tried the instructions from the Umami website, but that doesn't work at all (some npm errors about prisma)

As for backups, restoring should work. What was the issue there? Just clone the app from backup and check the logs?

The issue was that whenever I restored a backup, recent or from several weeks ago, the app still didn't start with the same errors in the logs.

Btw, what also worries me is that the restores from older backups always resulted in a none working app.
Shouldn't backups be a safety net to be able to roll back to a previous state where everything was working correctly?

Hi @girish , I considered that already but what's holding me back is that I'll lose all the site ID's (more than 30).
Isn't there any way I can try to manually run the migration script in recovery mode?