Issue Starting NodeBB on Cloudron After Server Restart

archos

Hi, after restarting the server today, I have an issue with NodeBB running on Cloudron. The application status shows "Not Responding", and the terminal in Cloudron UI is not working.

Is there any way to fix this issue? Thanks a lot for your help!

Jan 30 16:33:13 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:33:23 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:33:32 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:33:43 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:33:53 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:34:00 => Healtheck error: Error: connect ECONNREFUSED 172.18.18.198:4567
Jan 30 16:34:17 => Healtheck error: Error: Timeout of 7000ms exceeded
Jan 30 16:34:27 => Healtheck error: Error: Timeout of 7000ms exceeded
Jan 30 16:34:32 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:34:43 => Healtheck error: Error: connect EHOSTUNREACH 172.18.18.198:4567
Jan 30 16:33:58 2025-01-30T16:33:58+01:00
Jan 30 16:33:58 2025-01-30T16:33:58+01:00
Jan 30 16:33:58 2025-01-30T16:33:58+01:00
Jan 30 16:33:58 2025-01-30T16:33:58+01:00
Jan 30 16:33:58 DeprecationWarning: Collection.update() is deprecated. Use updateOne, updateMany, or bulkWrite.
Jan 30 16:33:58 Telemetry is now disabled.
Jan 30 16:33:58 Using Mongosh: 1.10.6
Jan 30 16:33:58 Using Mongosh: 1.10.6
Jan 30 16:33:58 ]0;mongosh Current Mongosh Log ID: 679b9be6007078407cd9800e
Jan 30 16:33:58 ]0;mongosh mongodb://<credentials>@mongodb:27017/6227182edf48e3d1?directConnection=trueCurrent Mongosh Log ID: 679b9be688bebc16ac8d5b07
Jan 30 16:33:58 => Bringing package.json up to speed
Jan 30 16:33:58 => Creating directories
Jan 30 16:33:58 => Setting up email
Jan 30 16:33:58 Connecting to: mongodb://<credentials>@mongodb:27017/6227182edf48e3d1?directConnection=true&appName=mongosh+1.10.6
Jan 30 16:33:58 For mongosh info see: https://docs.mongodb.com/mongodb-shell/
Jan 30 16:33:58 For mongosh info see: https://docs.mongodb.com/mongodb-shell/
Jan 30 16:33:58 Using MongoDB: 6.0.13
Jan 30 16:33:58 acknowledged: true,
Jan 30 16:33:58 insertedId: null,
Jan 30 16:33:58 matchedCount: 1,
Jan 30 16:33:58 modifiedCount: 0,
Jan 30 16:33:58 upsertedCount: 0
Jan 30 16:33:58 {
Jan 30 16:33:58 }
Jan 30 16:33:59 2025-01-30T16:33:59+01:00
Jan 30 16:33:59 2025-01-30T16:33:59+01:00
Jan 30 16:33:59 2025-01-30T16:33:59+01:00
Jan 30 16:33:59 2025-01-30T16:33:59+01:00
Jan 30 16:33:59
Jan 30 16:33:59 npm ERR!
Jan 30 16:33:59 npm ERR!
Jan 30 16:33:59 npm ERR!
Jan 30 16:33:59 npm ERR!
Jan 30 16:33:59 npm ERR!
Jan 30 16:33:59 npm ERR!
Jan 30 16:33:59 npm ERR! node_modules/nodebb-plugin-extended-markdown
Jan 30 16:33:59 npm ERR! node_modules/nodebb-plugin-markdown
Jan 30 16:33:59 npm ERR! node_modules/nodebb-plugin-markdown
Jan 30 16:33:59 npm ERR! peer nodebb-plugin-markdown@"^12.0.2" from nodebb-plugin-extended-markdown@2.0.1
Jan 30 16:33:59 npm ERR! nodebb-plugin-extended-markdown@"2.0.1" from the root project
Jan 30 16:33:59 npm ERR! nodebb-plugin-markdown@"13.0.0" from the root project
Jan 30 16:33:59 npm ERR! node_modules/nodebb-plugin-extended-markdown
Jan 30 16:33:59 npm ERR! peer nodebb-plugin-markdown@"^12.0.2" from nodebb-plugin-extended-markdown@2.0.1
Jan 30 16:33:59 npm ERR! nodebb-plugin-extended-markdown@"2.0.1" from the root project
Jan 30 16:33:59 npm ERR! /home/cloudron/.npm/_logs/2025-01-30T15_33_59_322Z-eresolve-report.txt
Jan 30 16:33:59 npm ERR! Conflicting peer dependency: nodebb-plugin-markdown@12.2.9
Jan 30 16:33:59 npm ERR! Could not resolve dependency:
Jan 30 16:33:59 npm ERR! Fix the upstream dependency conflict, or retry
Jan 30 16:33:59 npm ERR! For a full report see:
Jan 30 16:33:59 npm ERR! Found: nodebb-plugin-markdown@13.0.0
Jan 30 16:33:59 npm ERR! While resolving: nodebb-plugin-extended-markdown@2.0.1
Jan 30 16:33:59 npm ERR! this command with --force or --legacy-peer-deps
Jan 30 16:33:59 npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
Jan 30 16:33:59 npm ERR! ERESOLVE could not resolve
Jan 30 16:33:59 Error installing dependencies!
Jan 30 16:33:59 1. Bringing base dependencies up to date... started
Jan 30 16:33:59 1. Updating package.json file with defaults... OK
Jan 30 16:33:59 NodeBB Upgrade Complete!
Jan 30 16:33:59
Jan 30 16:33:59 [#########.........] / idealTree:code: timing idealTree:#root Completed in 1ms npm ERR! code ERESOLVE
Jan 30 16:33:59
Jan 30 16:33:59
Jan 30 16:33:59 Updating NodeBB...
Jan 30 16:33:59 Updating NodeBB...
Jan 30 16:33:59 npm ERR! A complete log of this run can be found in: /home/cloudron/.npm/_logs/2025-01-30T15_33_59_322Z-debug-0.log
Jan 30 16:33:59 2025-01-30T15:33:59.733Z [4567/54] - error: uncaughtException: Command failed: npm install --omit=dev
Jan 30 16:33:59 => Installing packages
Jan 30 16:33:59 Error occurred during upgrade: Error: Command failed: npm install --omit=dev
Jan 30 16:33:59 Error: Command failed: npm install --omit=dev
Jan 30 16:33:59 at /app/code/node_modules/commander/lib/command.js:1502:14
Jan 30 16:33:59 at /app/code/node_modules/commander/lib/command.js:1502:14 {"date":"Thu Jan 30 2025 15:33:59 GMT+0000 (Coordinated Universal Time)","error":{"output":[null,null,null],"pid":69,"signal":null,"status":1,"stderr":null,"stdout":null},"exception":true,"os":{"loadavg":[0.55,0.78,0.97],"uptime":33730.1},"process":{"argv":["/usr/local/node-18.18.0/bin/node","/app/code/nodebb","upgrade","--install"],"cwd":"/app/code","execPath":"/usr/local/node-18.18.0/bin/node","gid":1000,"memoryUsage":{"arrayBuffers":530315,"external":1195535,"heapTotal":21671936,"heapUsed":16562376,"rss":67280896},"pid":54,"uid":1000,"version":"v18.18.0"},"stack":"Error: Command failed: npm install --omit=dev\n at checkExecSyncError (node:child_process:890:11)\n at Object.execSync (node:child_process:962:15)\n at pkgInstall.installAll (/app/code/src/cli/package-install.js:129:9)\n at Object.handler (/app/code/src/cli/upgrade.js:22:19)\n at runSteps (/app/code/src/cli/upgrade.js:55:16)\n at Object.runUpgrade [as upgrade] (/app/code/src/cli/upgrade.js:98:9)\n at Command.<anonymous> (/app/code/src/cli/index.js:303:24)\n at Command.listener [as _actionHandler] (/app/code/node_modules/commander/lib/command.js:542:17)\n at /app/code/node_modules/commander/lib/command.js:1502:14","trace":[{"column":11,"file":"node:child_process","function":"checkExecSyncError","line":890,"method":null,"native":false},{"column":15,"file":"node:child_process","function":"Object.execSync","line":962,"method":"execSync","native":false},{"column":9,"file":"/app/code/src/cli/package-install.js","function":"pkgInstall.installAll","line":129,"method":"installAll","native":false},{"column":19,"file":"/app/code/src/cli/upgrade.js","function":"Object.handler","line":22,"method":"handler","native":false},{"column":16,"file":"/app/code/src/cli/upgrade.js","function":"runSteps","line":55,"method":null,"native":false},{"column":9,"file":"/app/code/src/cli/upgrade.js","function":"Object.runUpgrade [as upgrade]","line":98,"method":"runUpgrade [as upgrade]","native":false},{"column":24,"file":"/app/code/src/cli/index.js","function":null,"line":303,"method":null,"native":false},{"column":17,"file":"/app/code/node_modules/commander/lib/command.js","function":"Command.listener [as _actionHandler]","line":542,"method":"listener [as _actionHandler]","native":false},{"column":14,"file":"/app/code/node_modules/commander/lib/command.js","function":null,"line":1502,"method":null,"native":false}]}
Jan 30 16:33:59 at Command.<anonymous> (/app/code/src/cli/index.js:303:24)
Jan 30 16:33:59 at Command.<anonymous> (/app/code/src/cli/index.js:303:24)

joseph

It seems nodebb-plugin-markdown is causing some problem. Can you try disabling it ? https://docs.cloudron.io/apps/nodebb/#disabling-plugins . You might have to revert the app to a working state or put the app in recovery mode.

archos

@joseph said in Issue Starting NodeBB on Cloudron After Server Restart:

It seems nodebb-plugin-markdown is causing some problem. Can you try disabling it ? https://docs.cloudron.io/apps/nodebb/#disabling-plugins . You might have to revert the app to a working state or put the app in recovery mode.

I tried disabling the plugin, but I cannot type in the terminal. It looks like the application is frozen. I also tried enabling recovery mode, but the problem was not resolved.

archos

I tried running the command in recovery mode, and I got the following output in the terminal:

root@630e742c-440d-409a-97ef-45a5161aa337:/app/code# cd /app/code
root@630e742c-440d-409a-97ef-45a5161aa337:/app/code# ./nodebb reset -p nodebb-plugin-markdown
Warning: Failed to detect linux-musl, fallback to linux-gnu: The "offset" argument must be of type number. Received an instance of Object
2025-01-30T18:38:12.386Z [undefined/19] - error: uncaughtException: ENOENT: no such file or directory, mkdir 'logs'
Error: ENOENT: no such file or directory, mkdir 'logs'
    at Object.mkdirSync (node:fs:1398:3)
    at File._createLogDirIfNotExist (/app/code/node_modules/winston/lib/winston/transports/file.js:759:10)
    at new File (/app/code/node_modules/winston/lib/winston/transports/file.js:94:28)
    at Object.<anonymous> (/app/code/install/web.js:45:3)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18) {"date":"Thu Jan 30 2025 18:38:12 GMT+0000 (Coordinated Universal Time)","error":{"code":"ENOENT","errno":-2,"path":"logs","syscall":"mkdir"},"exception":true,"os":{"loadavg":[0.46,0.81,0.94],"uptime":44782.76},"process":{"argv":["/usr/local/node-18.18.0/bin/node","/app/code/nodebb","reset","-p","nodebb-plugin-markdown"],"cwd":"/app/code","execPath":"/usr/local/node-18.18.0/bin/node","gid":0,"memoryUsage":{"arrayBuffers":327376,"external":2265472,"heapTotal":30515200,"heapUsed":23418856,"rss":79994880},"pid":19,"uid":0,"version":"v18.18.0"},"stack":"Error: ENOENT: no such file or directory, mkdir 'logs'\n    at Object.mkdirSync (node:fs:1398:3)\n    at File._createLogDirIfNotExist (/app/code/node_modules/winston/lib/winston/transports/file.js:759:10)\n    at new File (/app/code/node_modules/winston/lib/winston/transports/file.js:94:28)\n    at Object.<anonymous> (/app/code/install/web.js:45:3)\n    at Module._compile (node:internal/modules/cjs/loader:1256:14)\n    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)\n    at Module.load (node:internal/modules/cjs/loader:1119:32)\n    at Module._load (node:internal/modules/cjs/loader:960:12)\n    at Module.require (node:internal/modules/cjs/loader:1143:19)\n    at require (node:internal/modules/cjs/helpers:119:18)","trace":[{"column":3,"file":"node:fs","function":"Object.mkdirSync","line":1398,"method":"mkdirSync","native":false},{"column":10,"file":"/app/code/node_modules/winston/lib/winston/transports/file.js","function":"File._createLogDirIfNotExist","line":759,"method":"_createLogDirIfNotExist","native":false},{"column":28,"file":"/app/code/node_modules/winston/lib/winston/transports/file.js","function":"new File","line":94,"method":null,"native":false},{"column":3,"file":"/app/code/install/web.js","function":null,"line":45,"method":null,"native":false},{"column":14,"file":"node:internal/modules/cjs/loader","function":"Module._compile","line":1256,"method":"_compile","native":false},{"column":10,"file":"node:internal/modules/cjs/loader","function":"Module._extensions..js","line":1310,"method":".js","native":false},{"column":32,"file":"node:internal/modules/cjs/loader","function":"Module.load","line":1119,"method":"load","native":false},{"column":12,"file":"node:internal/modules/cjs/loader","function":"Module._load","line":960,"method":"_load","native":false},{"column":19,"file":"node:internal/modules/cjs/loader","function":"Module.require","line":1143,"method":"require","native":false},{"column":18,"file":"node:internal/modules/cjs/helpers","function":"require","line":119,"method":null,"native":false}]}

Axtremus

I am observing similar issue after upgrading to NodeBB package version 2.14.x.
Among error logs I am seeing are "ERR!" logs about not being able to resolve the nodebb-plugin-emoji plugins, and the application being "not responding."

Had to downgrade to NodeBB package version 2.13.4 to resolve the error and get the application back up and running.

Funny thing is, if I "upgrade" only to package version 2.14.0, I get only "WARN" logs about not being able to resolve the nodebb-plugin-emoji plugins, and the application will eventually get up and run. But if I take one more step to upgrade to 2.14.1, then the "ERR!" logs come back and the application will be stuck "not responding" again. Downgrading from 2.14.1 back to 2.14.0 did not help -- I get the same "ERR!" logs as mentioned above and the application remains "not responding." I have to downgrade back to 2.13.4 to get out of this hole.

For now, I will disable auto-update and hope that smarter people figure this out and put out a revised NodeBB 2.14.x package that won't have the same or similar issue. (Not really sure how this works ... if 2.14.0 and 2.14.1 really have issues, I'm hoping they will be removed such that auto-update will skip over them and move on to a working update when one becomes available.)

joseph

Can you run /app/pkg/start.sh in the web terminal once and then do the nodebb reset ? The reason is that /app/pkg/start.sh will "initialize" the app with necessary run time configuration and directories.

hpz24

@joseph Hi, facing similar problems and tried your solution but it does not fix the issue.

nebulon

Looks like a dependency issue between plugins. Have you tried to disable some of the mentioned plugins? Maybe they need to be updated upstream and will start working in some time then.

See https://docs.cloudron.io/apps/nodebb/#disabling-plugins how to do this. Otherwise restore the app and only update once those plugins have a new fixed release.

hpz24

Disabling the plugins has not been successful: The main problem is, that the terminal stops working when nodebb crashes.
So, I fiddled around a bit more:
I´ve the same problem as @Axtremus:
I downgraded to version 2.13.4 and the app works. Then I updated to 2.14.0 and had also the WARN entries but the app works again. Then I restarted the app manually and the behaviour was the same as at the beginning: WARN entries changed to ERR entries and nothing works.
That means: The problem is in Version 2.14: The container works well till the next restart, then it breaks. Even if the app has daily backups, the backups are useless because the backuped a failed state.
Conclusion: I restored back to 2.13.4, disabled Auto-Update and have lost 9 days of posts and threads ..

joseph

@hpz24 said in Issue Starting NodeBB on Cloudron After Server Restart:

The main problem is, that the terminal stops working when nodebb crashes.

Right, so you need to use the recovery mode for this - https://docs.cloudron.io/apps/#recovery-mode . Put app in recovery mode and then if you run /app/pkg/start.sh it will set up the run time directories. After that, you can run nodebb cli and disable plugins.

(cc @girish , please put above in docs)

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Cloudron Forum

Issue Starting NodeBB on Cloudron After Server Restart