Add Pre-Backup and Post-Backup Hooks

[nj]

Cloudron doesn’t have a scheduler or cron system that can trigger a script before starting to backup an app, or after backup succeeds/fails. If we could configure a script to run at these times it would be nice.

• Before backup task starts
• After backup task succeeds
• After backup task fails

This would help some apps perform housekeeping. To be honest it would be much useful for the ERPNext app that I packaged.

[girish]

@nj said in Add Pre-Backup and Post-Backup Hooks:

This would help some apps perform housekeeping. To be honest it would be much useful for the ERPNext app that I packaged.

Can you tell me a bit more about this. Are we talking about some webhooks or some bash scripts to run ? If bash scripts, are these to be run on the host or inside the ERPNext container?

[nj]

@girish I'm thinking of running the script inside the container.

In my case, since ErpNext doesn't properly support MySQL and Postgres, a separate process of MariaDB is running in the container apart from the supervisor. The data generated by MariaDB resides at /app/data/mariadb folder so that it gets backed up regularly. The problem is, the backup task could run when the data is still being written, or in the middle of a database transaction.

If there was a pre-backup hook, I could place mariadb inside /run/mariadb and run mysqldump > /app/data/dump.sql right before backup so the database dump would be backed up instead of the whole mariadb folder, and the data consistency would be guaranteed.

I'd suggest the system waited for docker exec <app-container> bash -c /app/pkg/pre-backup-hook.sh before starting to back it up and ran docker exec <app-container> bash -c /app/pkg/post-backup-hook.sh it would be near perfect.

[girish]

@nj I guess we will also need recovery hooks in that case. I think it's probably easier to support MariaDB itself as an addon at that point.

So, ERPNext can never support MySQL is it? Do you know why it can only support MariaDB?

[robi]

@girish I'm pretty sure it does support both as a quick search reveals plenty of erpnext and mysql hits.

[nj]

Frappe devellopers tell that they officially only support MariaDB (their docs, tutorials, docker image, etc.) Hint: https://discuss.frappe.io/t/can-i-use-mysql-instead-of-mariadb/9078/2

Suppose, it supports MySQL. Even then, there is a major blocker because Frappe requirements are high:

• either the root user's password
  OR
• username/password for another user with permission to create new databases and users, and also delete them.

Possible Solution could be Cloudron MySQL's multiple databases. Could the Addon provide username/password for the user having these permissions?

• create new database
• create new user
• grant that user access to the new database
• when needed drop the database and the user
• some kind of limit to only allow that user to drop the user/database created by them would suffice.
• some kind of limit on the number of databases/users they can create would be cool.

So.. there's a huge possibility that Cloudon officially supports ErpNext, because MySQL is the only blocker now. I've already customized it to use Cloudron addons (redis, ldap, email is WIP).

Since ERPNext is one of the most voted and oldest entry, @girish or @nebulon could look into modifying MySQL addon to support database and user creation.

[jdaviescoates]

@nj said in Add Pre-Backup and Post-Backup Hooks:

y, @girish or @nebulon could look into modifying MySQL addon to support database and user creation.

Sounds to me like a MariaDB addon might be a better solution given what @girish said previously

@girish said in Add Pre-Backup and Post-Backup Hooks:

I think it's probably easier to support MariaDB itself as an addon

[girish]

@nj thanks for clarifying, indeed those database requirements are pretty far out. Wow, I am yet to see any web app software require database root credentials and create databases on the fly. Seems to not make sense.

[MooCloud_Matt]

@girish said in Add Pre-Backup and Post-Backup Hooks:

Seems to not make sense.

That's normal for ERPs, almost all ERPs require full access to their database.
This is mostly due to the Modular idea for modern ERP, which implements different DB for each module like a microservice.
And communicate between with API or Functions Hooks