Mattermost v8.0+ and PostgreSQL

[girish]

@chymian we have not made this move yet. It requires obsoleting the old package and moving to postgres. And yet, there is no easy way on mattermost side to switch databases. Essentially, we will have to abandon all our existing mattermost installation base. It's a difficult decision to make. Maintaining two separate packages while possible is just additional work for a single app. We have to make separate tests and packages for every mm release.

[chymian]

ok, thanks @girish,

Yes, the change will certainly not be easy and will require a lot of work on both sides.
This has been discussed for about a year now and there is still no solution in sight - not even for new installations.
The non acceptable burden here is, the additional effort to "new install" an outdated solution, which causes a additional work/data loss in switching systems whenever you (have to) change to postrgresql.

[cvachery]

@girish said in Mattermost v8.0+ and PostgreSQL:

And yet, there is no easy way on Mattermost side to switch databases.

I don't understand your remark as there is a whole page dedicated to the migration from MySQL to PostgreSQL : https://docs.mattermost.com/deploy/postgres-migration.html

I understand it is a decision which can be hard to take as it will make some users unhappy, but at some point you will have to make it because Mattermost is slowly but surely deprecating MySQL.

So the real question is not if you have to do the move or not but when you make it.
And IMHO the sooner the better :

• The more you wait, the more installations will be made on Cloudron using MySQL which will have to be migrated and therefore more users/customers will be impacted/annoyed
• People will move to something else than Cloudron for their install as PostgreSQL is required by more and more features.

[girish]

@cvachery I am totally with you. I find the migration guide quite complex. In many apps, one can just export and import using some portable format but instead in this guide we have to resort to become a db admin. Most Cloudron users have no experience with level of database administration and this is the source of issue.

I guess we have to just bite the bullet and migrate away and hope for the best

[Lanhild]

@girish What about freezing the MySQL package version (so that users are aware they'll have to install the PostgreSQL in the future) and promoting the PostgreSQL version instead.

That would allow the team to focus on only the PostgreSQL package.

My idea;

• Delist the MySQL version from the app store
• Add the new PostgreSQL version, possibly with a note in the manifest description explaining the why

Users are able to keep their existing app with the MySQL version, but would be directed towards moving to the new, maintained PostgreSQL version.

[girish]

Alright, I will make a new package that uses postgres. The previous version will be delisted. We will maintain it for a bit until it gets deprecated entirely by mattermost upstream.

[girish]

New package is published now. I tried the export/import instead of db approach and it works well. The latest version of mattermost has broken import though, I have to report this upstream.

Some docs at https://docs.cloudron.io/apps/mattermost/#migrate-to-postgres if someone wants to try the migration. I haven't tried the database migration yet.

[girish]

Reported the import issue at https://github.com/mattermost/mattermost/issues/27182

[cvachery]

Excellent! I will try to migrate an external mattermost with postgresql to Cloudron

[girish]

@cvachery I just managed to do a database migration using pgloader. Just publishing my notes in the docs.

[cvachery]

@girish Awesome, I will wait for your doc then see if I can validate it

[girish]

@cvachery done. https://docs.cloudron.io/apps/mattermost/#database-migration

[Lanhild]

I'll try and report

[cvachery]

I've mostly done it using the official mattermost documentation : https://docs.mattermost.com/onboard/migrating-to-mattermost.html#migrate-mattermost-from-one-server-to-another and https://docs.mattermost.com/deploy/backup-disaster-recovery.html#backup

Copied the config.json from the origin server and changer the SQL settings with the logins from the cloudron mattermost file.

Only issue I'm having is the importing of the local data to cloudron. I've around 20GB of data to be moved to the files folder but the upload crash when uploading the tar.gz file. Uploading the file uncompressed crash too after a while (couple of hours)

[Lanhild]

@cvachery said in Mattermost v8.0+ and PostgreSQL:

Only issue I'm having is the importing of the local data to cloudron. I've around 20GB of data to be moved to the files folder but the upload crash when uploading the tar.gz file. Uploading the file uncompressed crash too after a while (couple of hours)

Try with the cloudron CLI, it might work better

[girish]

@cvachery The data directories of apps are at /home/yellowtent/appsdata/{appid}/data . Just cp from one dir to another. The appid you can figure by looking at the URL bar or in the Updates section.

[cvachery]

Sometimes I do forget that Cloudron isn't a blackbox SaaS application and that I do have access to the server
Currently doing an rsync to the server will let you know when it's all done.
Thank you

[cvachery]

Ok So it's perfectly working.

Steps done :

• Dumped the PostgreSQL DB and config.json file
• Uploaded them to the cloudron server
• Copied the files folder to the cloudron server
• Imported the dump with psql from the app terminal
• Changed the files ownership to cloudron in the file manager
• Edited in the config.json from the origin server to add the new files path from the cloudron config.json
  • "ServiceSettings" => "SiteURL"
  • "SqlSettings" => "DataSource" and "DataSourceReplicas"
  • "FileSettings" => "Directory"

[Lanhild]

@cvachery Do you have a list of commands/queries you've done?

I tried both methods, and had issues with them.

The 1st (database migration) renders in a plethora of errors with indexes and columns. Even though I followed Cloudron docs and the upstream's, I can't seem to be able to make it work.

The 2nd (Bulk import) didn't restore any of my DMs which makes it a no-go.

If you could provide some guidance, I'd be very grateful.

[Lanhild]

@Lanhild said in Mattermost v8.0+ and PostgreSQL:

@cvachery Do you have a list of commands/queries you've done?

I tried both methods, and had issues with them.

The 1st (database migration) renders in a plethora of errors with indexes and columns. Even though I followed Cloudron docs and the upstream's, I can't seem to be able to make it work.

The 2nd (Bulk import) didn't restore any of my DMs which makes it a no-go.

If you could provide some guidance, I'd be very grateful.

@cvachery If you can get back to me on this, I'd be very happy.