dolibarr - ERP & CRM for Business

[girish]

Oh wow that was quick! Thanks @erics. Do you have the tests already as well? It's a requirement we have to get things published in the app store. It helps us keep things updated.

I wrote some instructions at https://forum.cloudron.io/topic/2850/writing-automated-tests-for-packages on how to go about this. They are fairly easy to write if you know a bit of node.

[erics]

@girish ooooops no tests for the moment, test directory include moodle stuff

i will have a look maybe tomorrow

[girish]

@erics I forked the project to https://git.cloudron.io/cloudron/dolibarr-app and added you to the project. Feel free to push there directly.

I am working on the moodle tests myself as we speak, just pushed the first iteration of tests of moodle.

[erics]

@girish is there a special envvar during UPGRADE or INSTALL-BACKUP ?

because i launch database setup in start.sh ... and this is not needed if we restore backup.

thanks

[nebulon]

There is no env var injected in such cases. You would have to somehow figure out if the db setup was already done or not. Often this is done via a state file like https://git.cloudron.io/cloudron/wordpress-app/-/blob/master/start.sh#L39

[erics]

@nebulon thanks, i will do a test with that (but i already have a file-flag), ... is data backup restored "before" start.sh is launched ?

i will make a test

i have an other question, how could a do a loop in tests.js like that pseudo code:

Note: i have a message on index.php "database install in progress please come back later"

so here is what i try to do in test.js

while(web index contains "database setup in progress")
sleep(300)
loop

[girish]

@erics said in dolibarr - ERP & CRM for Business:

@nebulon thanks, i will do a test with that (but i already have a file-flag), ... is data backup restored "before" start.sh is launched ?

Yes. When a restore is done, start.sh is called after the database is restore and all the files in /app/data are already in place. This is why you don't really need to know if it's upgrade/install/backup. In the case of your code, that install.lock at https://git.cloudron.io/cloudron/dolibarr-app/-/blob/master/start.sh#L14 will already be present. And thus, the whole install block will get skipped when an app is restored.

[girish]

@erics said in dolibarr - ERP & CRM for Business:

Note: i have a message on index.php "database install in progress please come back later"
so here is what i try to do in test.js
while(web index contains "database setup in progress")
sleep(300)
loop

I think for the tests at least, it's fine to just sleep(20-30 seconds or whatever is for sure to work). Just do something like:

function install(done) {
    execSync('cloudron install ...');
    console.log('waiting for 30 seconds for database setup to complete');
    setTimeout(done, 30 * 1000);
}

Regardless of the tests, if you have a "healthCheckPath" which is more reliable i.e it returns 2xx or 3xx http status only after db setup is complete and app is ready to use then you can put that in https://git.cloudron.io/cloudron/dolibarr-app/-/blob/master/CloudronManifest.json#L8 . If you have such a route in the app, then you don't need the sleep above. This also has the advantage that the user will see 'Running' in cloudron dashboard only when it's ready to use (otherwise, right now, he will see db is getting setup message).

[erics]

@girish so nice

for the moment i have an other problem with "App restore error: Installation failed: Cannot download from noop backend"

Oct 07 23:39:56 box:shell clearVolume spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/clearvolume.sh clear /home/yellowtent/appsdata/773326e5-e8f3-490c-8147-76c35839873f/data
Oct 07 23:39:56 box:backups download: Downloading 2020-10-07-213836-698/app_92291923-34a2-4ca0-9637-430e75e3f679_2020-10-07-213838-779_v0.1.3 of format tgz to {"localRoot":"/home/yellowtent/appsdata/773326e5-e8f3-490c-8147-76c35839873f","layout":[]}
Oct 07 23:39:56 box:storage/noop download: 2020-10-07-213836-698/app_92291923-34a2-4ca0-9637-430e75e3f679_2020-10-07-213838-779_v0.1.3.tar.gz
Oct 07 23:39:23 Fresh installation, performing Dolibarr first time setup
Oct 07 23:39:23 AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.8. Set the 'ServerName' directive globally to suppress this message
Oct 07 23:39:23 - STEP N°1 ...
Oct 07 23:39:24 [done]
Oct 07 23:39:24 - STEP N°2 ...
Oct 07 23:39:24 [done]
Oct 07 23:39:24 - STEP N°3 ...
Oct 07 23:40:16 box:storage/noop download: 2020-10-07-213836-698/app_92291923-34a2-4ca0-9637-430e75e3f679_2020-10-07-213838-779_v0.1.3.tar.gz
Oct 07 23:40:36 box:storage/noop download: 2020-10-07-213836-698/app_92291923-34a2-4ca0-9637-430e75e3f679_2020-10-07-213838-779_v0.1.3.tar.gz
Oct 07 23:40:56 box:storage/noop download: 2020-10-07-213836-698/app_92291923-34a2-4ca0-9637-430e75e3f679_2020-10-07-213838-779_v0.1.3.tar.gz
Oct 07 23:41:16 box:storage/noop download: 2020-10-07-213836-698/app_92291923-34a2-4ca0-9637-430e75e3f679_2020-10-07-213838-779_v0.1.3.tar.gz
Oct 07 23:41:16 box:backups downloadApp: time: 80.01
Oct 07 23:41:16 box:apptask test.xxxxxxxxxxx.fr error installing app: BoxError: Cannot download from noop backend
Oct 07 23:41:16 box:apptask test.xxxxxxxxxxx.fr updating app with values: {"installationState":"error","error":{"message":"Cannot download from noop backend","reason":"Not implemented","taskId":"93","installationState":"pending_restore"}}
Oct 07 23:41:16 box:taskworker Task took 103.994 seconds
Oct 07 23:41:16 box:tasks setCompleted - 93: {"result":null,"error":{"stack":"BoxError: Cannot download from noop backend\n at Object.download (/home/yellowtent/box/src/storage/noop.js:59:14)\n at /home/yellowtent/box/src/backups.js:774:40\n at Timeout.retryAttempt [as _onTimeout] (/home/yellowtent/box/node_modules/async/dist/async.js:4611:9)\n at ontimeout (timers.js:436:11)\n at tryOnTimeout (timers.js:300:5)\n at listOnTimeout (timers.js:263:5)\n at Timer.processTimers (timers.js:223:10)","name":"BoxError","reason":"Not implemented","details":{},"message":"Cannot download from noop backend"}}
Oct 07 23:41:16 box:tasks 93: {"percent":100,"result":null,"error":{"stack":"BoxError: Cannot download from noop backend\n at Object.download (/home/yellowtent/box/src/storage/noop.js:59:14)\n at /home/yellowtent/box/src/backups.js:774:40\n at Timeout.retryAttempt [as _onTimeout] (/home/yellowtent/box/node_modules/async/dist/async.js:4611:9)\n at ontimeout (timers.js:436:11)\n at tryOnTimeout (timers.js:300:5)\n at listOnTimeout (timers.js:263:5)\n at Timer.processTimers (timers.js:223:10)","name":"BoxError","reason":"Not implemented","details":{},"message":"Cannot download from noop backend"}}

[erics]

@girish healthCheckPath is PERFECT !

[girish]

@erics said in dolibarr - ERP & CRM for Business:

for the moment i have an other problem with "App restore error: Installation failed: Cannot download from noop backend"

Looks like the Cloudron server which you are installing/testing this on has backups disabled! Go to Backups -> change to file system or something. Without backups, one cannot restore.

[erics]

@girish yes (i'm) so stupid i just found it now

[erics]

New question : does ".lock" files could be excluded from backups ?

when dolibarr install is complete a install.lock file is created, but when cloudron backup is restaured this file is missing ...

[erics]

ok, i switched from install.lock to .dbsetup file like wordpress and hoooraaa it works

new problem during autotest: database error on test2 + restoring data ... next step tomorrow !

[erics]

00:41 $ USERNAME=cloudronusername PASSWORD=cloudronpassword node_modules/.bin/mocha --bail test.js

  Application life cycle test
    - build app
    installation and configuration
App is being installed.

 => Queued 
 => Cleaning up old install 
 => Downloading image ..........
 => Creating container .
 => Wait for health check .........................................

App is installed.
      ✓ install app (61950ms)
      ✓ can get app information (475ms)
      ✓ can view welcome page (812ms)
      ✓ can login (2045ms)

 => Queued 
 => Snapshotting app test.example.com 
 => Uploading app snapshot test.example.com 
 => Uploading backup 1M@1MBps (test.example.com) 

App is backed up
      ✓ backup app (5252ms)

 => Waiting for app to be uninstalled 
 => Queued 
 => Deleting container 
 => Teardown addons ...
 => Unregistering domains 

App test.example.com successfully uninstalled.
App is being installed.

 => Queued 
 => Cleaning up old install 
 => Downloading image .........
 => Creating container ..
 => Wait for health check ............................................................

App is installed.

 => Queued 
 => Cleaning up old install 
 => Registering subdomains 
 => Downloading image 
 => Download backup and restoring addons ........
 => Downloading 1M@1MBps ...................
 => Creating container 
 => Waiting for DNS propagation 
 => Wait for health check 

App is restored
      ✓ restore app (131164ms)

 => Queued 
 => Cleaning up old install .
 => Registering subdomains .
 => Creating container 
 => Wait for health check ..

App configured
      ✓ move to different location (10705ms)
      ✓ can access dashboard (508ms)

 => Waiting for app to be uninstalled 
 => Queued 
 => Deleting container 
 => Teardown addons ........
 => Deleting image 
 => Unregistering domains 

App test2.example.com successfully uninstalled.
      ✓ uninstall app (15220ms)
    update
Failed to get app info from store: 404 message: No such app
      1) can install app


  9 passing (4m)
  1 pending
  1 failing

  1) Application life cycle test
       update
         can install app:
     Error: Command failed: cloudron install --appstore-id org.dolibarr.cloudronapp --location test
      at checkExecSyncError (child_process.js:630:11)
      at execSync (child_process.js:666:15)
      at Context.<anonymous> (test.js:195:13)
      at processImmediate (internal/timers.js:456:21)

[girish]

@erics Great progress so far! I guess we need to publish the app to the appstore for the update test to work. Let me do that right now.

BTW, just a quick note: If there is some upgrade script that needs to be run, we have to do it in the else clause of https://git.cloudron.io/cloudron/dolibarr-app/-/blob/master/start.sh#L14 . Or does dolibarr automatically update/run new db migrations on first run of new version?

[girish]

@erics It's published as unstable now. You can try the update test now.

[erics]

@girish excellent, i will continue a bit later.

just a note about git (https://git.cloudron.io/cloudron/dolibarr-app) : i don't have write access so i made a merge request wich is "Ready to be merged automatically. Ask someone with write access to this repository to merge this request "

[erics]

@girish that's pleasant :

00:45 $ USERNAME=cloudronusername PASSWORD=cloudronpassword node_modules/.bin/mocha --bail test.js
Application life cycle test
- build app
installation and configuration
App is being installed.

=> Queued
=> Cleaning up old install
=> Downloading image ........................
=> Creating container .
=> Wait for health check ......................................

App is installed.
✓ install app (74925ms)
✓ can get app information (566ms)
✓ can view welcome page (860ms)
✓ can login (2159ms)

=> Queued
=> Snapshotting app test.example.com
=> Uploading app snapshot test.example.com

App is backed up
✓ backup app (4109ms)

=> Waiting for app to be uninstalled
=> Queued
=> Deleting container
=> Teardown addons ...
=> Deleting image

App test.example.com successfully uninstalled.
App is being installed.

=> Queued
=> Cleaning up old install
=> Downloading image .........
=> Creating container .
=> Configuring reverse proxy
=> Wait for health check ................................

App is installed.

=> Queued
=> Cleaning up old install
=> Registering subdomains
=> Downloading image
=> Download backup and restoring addons ..
=> Downloading 1M@1MBps .........
=> Creating container .
=> Wait for health check ........

App is restored
✓ restore app (92784ms)

=> Queued
=> Cleaning up old install .
=> Creating container
=> Waiting for DNS propagation
=> Wait for health check ..

App configured
✓ move to different location (9759ms)
✓ can access dashboard (477ms)

=> Waiting for app to be uninstalled
=> Queued
=> Deleting container
=> Teardown addons ...
=> Unregistering domains

App test2.example.com successfully uninstalled.
✓ uninstall app (8769ms)
update
App is being installed.

=> Queued
=> Downloading icon
=> Registering subdomains
=> Downloading image
=> Creating container
=> Wait for health check ..

App is installed.
✓ can install app (11025ms)
✓ can get app information (564ms)
✓ can view welcome page (308ms)

=> Waiting for app to be uninstalled
=> Queued
=> Deleting container ..........
=> Teardown addons ......
=> Unregistering domains

App test.example.com successfully uninstalled.
✓ uninstall app (22671ms)

13 passing (4m)
1 pending

[luckow]

Thank you for the packaging of dolibarr.
My first 2 cents: let us define, that first language has to be english. It took me around 5 minutes to find out where I can switch from french to english