Custom Apps

[BrutalBirdie]

https://git.cloudron.io/cloudron/nextcloud-app

[paul.toone]

@mehdi Just to make sure I understand correctly, in the manifest in the addons section, where it has "postgresql": {} would I just change that to "mysql": {} and then change the config file options for nextcloud? Will changing that value automatically create a database and user in the mysql container?

[A Former User]

@paul-toone correct. You would access the user and db info via the env vars listed in that first link @mehdi sent.

[paul.toone]

Perfect. Makes sense, thank you!

[BrutalBirdie]

I would advise against what you are planing to do.

Just to make sure I understand correctly, in the manifest in the addons section, where it has "postgresql": {} would I just change that to "mysql": {} and then change the config file options for nextcloud? Will changing that value automatically create a database and user in the mysql container?

Yes adding mysql to the addons would create a database and user but you would not be done there.
Since the whole Nextcloud app is build for postgresql.

Just an example from the start.sh

tables=$(PGPASSWORD=${CLOUDRON_POSTGRESQL_PASSWORD} psql -h ${CLOUDRON_POSTGRESQL_HOST} -p ${CLOUDRON_POSTGRESQL_PORT} -U ${CLOUDRON_POSTGRESQL_USERNAME} -d ${CLOUDRON_POSTGRESQL_DATABASE} -t -c "SELECT table_name FROM information_schema.tables WHERE table_schema='public';")

    for table in ${tables}; do
        if [[ $table != oc_* ]]; then
            echo "${table} -> oc_${table}"
            PGPASSWORD=${CLOUDRON_POSTGRESQL_PASSWORD} psql -h ${CLOUDRON_POSTGRESQL_HOST} -p ${CLOUDRON_POSTGRESQL_PORT} -U ${CLOUDRON_POSTGRESQL_USERNAME} -d ${CLOUDRON_POSTGRESQL_DATABASE} -c "ALTER TABLE ${table} RENAME TO oc_${table}"
        fi
    done

You would have to look into the app and build a new one for your use-case with mysql etc.

But what is the main goal? Migrating Data from A>B?
I would rather do that with Nextcloud Sync and have a working Nextcloud App.

To be honest this way you end up with a custom app which you will have to maintain yourself.
With the official one you can count on many people to push updates and improve everything.

[A Former User]

@brutalbirdie oh right... yes he's have to modify the start.sh as well. But yes using the packaged version is going to be your best bet.

[paul.toone]

@BrutalBirdie Yes, I knew there were some additional configurations that would need to be done but maybe nextcloud sync would be the best bet for now.

However, we are very impressed with cloudron and are looking at purchasing a license and moving a handful of our servers to this platform so I figured this would be good to learn how to create custom apps for platforms that are not currently on the app store. I'm excited to get into the guts of this so if NextCloud Sync works, I'll be back I'm sure when I work on some custom apps in the future. The support form is awesome with a lot of good responses that come very quickly. Thank you all and I'll be back if NextCloud Sync doesn't work.

[paul.toone]

@brutalbirdie I'll start looking into it but would nextcloud sync work between two servers with different databases? One as MySQL and one PostgreSQL

[BrutalBirdie]

@atrilahiji I doubt it being done with only manifest and start.sh

If I understand @paul-toone (putting together information from is posts) gitlab-ee.
He wants or is using cloudron for his company or dev team.

I would not want a custom and self maintained app for Nextcloud

May sound rude but with the official one I can complain if something is not working as intended.

But now imagine the chaos when a custom build Nextcloud has a problem and some Data is lost. (worst case)

. . .

When using Cloudron for a company or a team I want a stable environment which I can rely on.

[paul.toone]

@brutalbirdie You are correct. This is for a small business. The main reason is that I just moved everyone off of OwnCloud right before Covid. If I just started with a maintained image and asked everyone to move from one nextcloud server to another I will get a lot of pushback. If NextCloud Sync will in fact sync from a completely different server setup using mysql as db backend to this postgres app I will be extremely happy!

[BrutalBirdie]

@paul-toone Hmm

The main reason is that I just moved everyone off of OwnCloud right before Covid. If I just started with a maintained image and asked everyone to move from one nextcloud server to another I will get a lot of pushback. If NextCloud Sync will in fact sync from a completely different server setup using mysql as db backend to this postgres app I will be extremely happy!

https://nextcloud.com/install/#

Nextcloud Sync is for syncing data from the Nextcloud to your local device.
It's not meant exactly for what you want, but could be abused as a migration tool

My thought process was like. . .

Why not tell everyone to install Nextcloud Sync, Sync Data to their local machine and then change the Sync Client to the new Server.
So now the Sync client will upload the 'old' data to the new server.

There is still another option.
https://docs.nextcloud.com/server/15/admin_manual/maintenance/migrating.html

Still there is the mysql>postgresql 'issue'.
When exporting from mysql you could use the compatibility argument aka

--compatible=postgresql

And try importing this into the new Nextcloud inside Cloudron.

This way you can try a migration from OLD Nextcloud with a Backup to Cloudron.

[paul.toone]

@brutalbirdie fantastic advice. I'll give it a shot and see how it goes. Thanks for your help

[BrutalBirdie]

@paul-toone I thought a little about the import old DB.
This will also import old users (I suppose?) and cloudron uses LDAP and I assume your old Nextcloud did not?
Now "mapping" the old user/data to the Cloudron LDAP User?

I don't know how much is migrated from this but I fear when doing the DB import there are other things that might break as well.

But if you do that, be sure to document your pain and findings here so other people in the future can benefit from your example.

[paul.toone]

@brutalbirdie Yeah, I'm concerned about this as well. I'll definitely keep notes on the progress

[paul.toone]

Just an update, I was able to get NextCloud converted over. Dumping the existing mysql database with PostgreSQL compatibility did not work. I'm next going to test out user mapping via cloudron LDAP and once that is complete I'll update all my notes into a more readable format. But the process was essentially this:

• Have a fresh NextCloud App running on Cloudron and immediately enter recovery mode

• Bring up a temporary machine running PostgreSQL

• Build pgloader on the temporary PostgreSQL Server

  • sudo apt install sbcl unzip libsqlite3-dev gawk curl make freetds-dev libzip-dev
  • curl -fsSLO https://github.com/dimitri/pgloader/archive/v3.6.2.tar.gz
  • tar xvf v3.6.2.tar.gz
  • cd pgloader-3.6.2/
  • make pgloader
  • sudo mv ./build/bin/pgloader /usr/local/bin/

• Create a postgres user

  • sudo -u postgres createuser --interactive -P
  • <NewPostgresUser>(username)
  • Set the password for this role <NewPostgresPassword>
  • Make the user a super user

• Create an empty postgres DB

  • sudo -u postgres createdb <emptyDBName>

• Create a user on the MySQLDB and give it all privileges to the NextCloud DB

  • mysql> CREATE USER ‘<someMySQLUserName>’@’<postgersServerIP’ IDENTIFIED BY ‘<SomeMySQLUserPassword>’
  • mysql> GRANT ALL ON <nextcloudDB>.* to ‘<someMySQLUserName>’@’<postgresServerIP>’;
  • mysql> flush privileges;

• From the PostgreSQL Server ran pgloader

  • pgloader mysql://<someMySQLUserName>:<SomeMySQLUserPassword>@mysql_server_ip/<nextcloudDB> postgresql://<NewPostgresUser>:<NewPostgresPassword>@localhost/<emptyDBName>

• Dump the PostgreSQL Database

  • pg_dump <emptyDBName> > pgdump.sql

• Get CLOUDRON_POSTGRESQL_USERNAME from the app container running the NextCloud APP.

• sed the .sql file and replace the schema with public

• sed the .sql file and replace the <table owners> with CLOUDRON_POSTGRESQL_USERNAME

• Open a web terminal to the NextCloud App that is in recovery mode and upload the .sql file via the Upload to /tmp button

• Run the following commands grabbed from https://docs.cloudron.io/guides/import-postgresql/

  • sed -e 's/CREATE EXTENSION/-- CREATE EXTENSION/g' -e 's/COMMENT ON EXTENSION/-- COMMENT ON EXTENSION/g' /tmp/pgdump.sql > /tmp/pgdump_mod.sql
  • PGPASSWORD=${CLOUDRON_POSTGRESQL_PASSWORD} psql -h ${CLOUDRON_POSTGRESQL_HOST} -p ${CLOUDRON_POSTGRESQL_PORT} -U ${CLOUDRON_POSTGRESQL_USERNAME} -d ${CLOUDRON_POSTGRESQL_DATABASE} -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public"
  • PGPASSWORD=${CLOUDRON_POSTGRESQL_PASSWORD} psql -h ${CLOUDRON_POSTGRESQL_HOST} -p ${CLOUDRON_POSTGRESQL_PORT} -U ${CLOUDRON_POSTGRESQL_USERNAME} -d ${CLOUDRON_POSTGRESQL_DATABASE} --set ON_ERROR_STOP=on --file=/tmp/pgdump_mod.sql

• Move just the data from the old NextCloud server into the data directory of the NextCloud App on Cloudron. This is found in the app’s config page | Storage Section, should be something like:
  /home/yellowtent/appsdata/<containerHash>/data

• Bring the container out of recovery and boot up

Again, I'll update these notes if/when I try again and get cloudron's LDAP to map to the users in the database but for now, maybe these steps will help someone looking to convert and existing installation over to a Cloudron app.

[paul.toone]

One other note, the .sql file originally failed to upload due to the dump having the following line:

Create SCHEMA public;

I commented that out and the import ran fine. I'm guessing there is a command to dump the Postgres DB without a create statement but someone smarter than me probably knows it. I haven't used Postgres a day in my life until yesterday so I'm still getting to know it. I have only used MySQL, Maria and MS SQL and all of those have an option to dump without the create statement, but I figured it was just easier to comment that line out of the SQL file instead of looking up the command and doing a new dump.