Mattermost v8.0+ and PostgreSQL

[girish]

@cvachery I am totally with you. I find the migration guide quite complex. In many apps, one can just export and import using some portable format but instead in this guide we have to resort to become a db admin. Most Cloudron users have no experience with level of database administration and this is the source of issue.

I guess we have to just bite the bullet and migrate away and hope for the best

[Lanhild]

@girish What about freezing the MySQL package version (so that users are aware they'll have to install the PostgreSQL in the future) and promoting the PostgreSQL version instead.

That would allow the team to focus on only the PostgreSQL package.

My idea;

• Delist the MySQL version from the app store
• Add the new PostgreSQL version, possibly with a note in the manifest description explaining the why

Users are able to keep their existing app with the MySQL version, but would be directed towards moving to the new, maintained PostgreSQL version.

[girish]

Alright, I will make a new package that uses postgres. The previous version will be delisted. We will maintain it for a bit until it gets deprecated entirely by mattermost upstream.

[girish]

New package is published now. I tried the export/import instead of db approach and it works well. The latest version of mattermost has broken import though, I have to report this upstream.

Some docs at https://docs.cloudron.io/apps/mattermost/#migrate-to-postgres if someone wants to try the migration. I haven't tried the database migration yet.

[girish]

Reported the import issue at https://github.com/mattermost/mattermost/issues/27182

[cvachery]

Excellent! I will try to migrate an external mattermost with postgresql to Cloudron

[girish]

@cvachery I just managed to do a database migration using pgloader. Just publishing my notes in the docs.

[cvachery]

@girish Awesome, I will wait for your doc then see if I can validate it

[girish]

@cvachery done. https://docs.cloudron.io/apps/mattermost/#database-migration

[Lanhild]

I'll try and report

[cvachery]

I've mostly done it using the official mattermost documentation : https://docs.mattermost.com/onboard/migrating-to-mattermost.html#migrate-mattermost-from-one-server-to-another and https://docs.mattermost.com/deploy/backup-disaster-recovery.html#backup

Copied the config.json from the origin server and changer the SQL settings with the logins from the cloudron mattermost file.

Only issue I'm having is the importing of the local data to cloudron. I've around 20GB of data to be moved to the files folder but the upload crash when uploading the tar.gz file. Uploading the file uncompressed crash too after a while (couple of hours)

[Lanhild]

@cvachery said in Mattermost v8.0+ and PostgreSQL:

Only issue I'm having is the importing of the local data to cloudron. I've around 20GB of data to be moved to the files folder but the upload crash when uploading the tar.gz file. Uploading the file uncompressed crash too after a while (couple of hours)

Try with the cloudron CLI, it might work better

[girish]

@cvachery The data directories of apps are at /home/yellowtent/appsdata/{appid}/data . Just cp from one dir to another. The appid you can figure by looking at the URL bar or in the Updates section.

[cvachery]

Sometimes I do forget that Cloudron isn't a blackbox SaaS application and that I do have access to the server
Currently doing an rsync to the server will let you know when it's all done.
Thank you

[cvachery]

Ok So it's perfectly working.

Steps done :

• Dumped the PostgreSQL DB and config.json file
• Uploaded them to the cloudron server
• Copied the files folder to the cloudron server
• Imported the dump with psql from the app terminal
• Changed the files ownership to cloudron in the file manager
• Edited in the config.json from the origin server to add the new files path from the cloudron config.json
  • "ServiceSettings" => "SiteURL"
  • "SqlSettings" => "DataSource" and "DataSourceReplicas"
  • "FileSettings" => "Directory"

[Lanhild]

@cvachery Do you have a list of commands/queries you've done?

I tried both methods, and had issues with them.

The 1st (database migration) renders in a plethora of errors with indexes and columns. Even though I followed Cloudron docs and the upstream's, I can't seem to be able to make it work.

The 2nd (Bulk import) didn't restore any of my DMs which makes it a no-go.

If you could provide some guidance, I'd be very grateful.

[Lanhild]

@Lanhild said in Mattermost v8.0+ and PostgreSQL:

@cvachery Do you have a list of commands/queries you've done?

I tried both methods, and had issues with them.

The 1st (database migration) renders in a plethora of errors with indexes and columns. Even though I followed Cloudron docs and the upstream's, I can't seem to be able to make it work.

The 2nd (Bulk import) didn't restore any of my DMs which makes it a no-go.

If you could provide some guidance, I'd be very grateful.

@cvachery If you can get back to me on this, I'd be very happy.

[cvachery]

@Lanhild Sorry missed your message.
I will see if I can do another test migration this afternoon (CEST) and take some more notes. But basically I wrote the steps below on a clean Mattermost PostgreSQL install:

• Dumped the DB with : pg_dump -d mattermost > mattermost.dump
• Renamed on cloudron config.json to config.json.orig
• Copied mattermost.dump and config.json to cloudron mattermost via the file manager
• Edited config.json with the value from config.json.orig (to get the PostgreSQL details)
  • "ServiceSettings" => "SiteURL"
  • "SqlSettings" => "DataSource" and "DataSourceReplicas"
  • "FileSettings" => "Directory"

• From the terminal app did PASSWORD=${CLOUDRON_POSTGRESQL_PASSWORD} pg_restore -h ${CLOUDRON_POSTGRESQL_HOST} -p ${CLOUDRON_POSTGRESQL_PORT} -U ${CLOUDRON_POSTGRESQL_USERNAME} -d ${CLOUDRON_POSTGRESQL_DATABASE} ../data/mattermost.dump

• Rsynced all my attachments from origin_server to cloudron_server:app_folder (cf girish post )
• Restarted the app