Issue Starting NodeBB on Cloudron After Server Restart

[archos]

@joseph said in Issue Starting NodeBB on Cloudron After Server Restart:

It seems nodebb-plugin-markdown is causing some problem. Can you try disabling it ? https://docs.cloudron.io/apps/nodebb/#disabling-plugins . You might have to revert the app to a working state or put the app in recovery mode.

I tried disabling the plugin, but I cannot type in the terminal. It looks like the application is frozen. I also tried enabling recovery mode, but the problem was not resolved.

[archos]

I tried running the command in recovery mode, and I got the following output in the terminal:

root@630e742c-440d-409a-97ef-45a5161aa337:/app/code# cd /app/code
root@630e742c-440d-409a-97ef-45a5161aa337:/app/code# ./nodebb reset -p nodebb-plugin-markdown
Warning: Failed to detect linux-musl, fallback to linux-gnu: The "offset" argument must be of type number. Received an instance of Object
2025-01-30T18:38:12.386Z [undefined/19] - error: uncaughtException: ENOENT: no such file or directory, mkdir 'logs'
Error: ENOENT: no such file or directory, mkdir 'logs'
    at Object.mkdirSync (node:fs:1398:3)
    at File._createLogDirIfNotExist (/app/code/node_modules/winston/lib/winston/transports/file.js:759:10)
    at new File (/app/code/node_modules/winston/lib/winston/transports/file.js:94:28)
    at Object.<anonymous> (/app/code/install/web.js:45:3)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18) {"date":"Thu Jan 30 2025 18:38:12 GMT+0000 (Coordinated Universal Time)","error":{"code":"ENOENT","errno":-2,"path":"logs","syscall":"mkdir"},"exception":true,"os":{"loadavg":[0.46,0.81,0.94],"uptime":44782.76},"process":{"argv":["/usr/local/node-18.18.0/bin/node","/app/code/nodebb","reset","-p","nodebb-plugin-markdown"],"cwd":"/app/code","execPath":"/usr/local/node-18.18.0/bin/node","gid":0,"memoryUsage":{"arrayBuffers":327376,"external":2265472,"heapTotal":30515200,"heapUsed":23418856,"rss":79994880},"pid":19,"uid":0,"version":"v18.18.0"},"stack":"Error: ENOENT: no such file or directory, mkdir 'logs'\n    at Object.mkdirSync (node:fs:1398:3)\n    at File._createLogDirIfNotExist (/app/code/node_modules/winston/lib/winston/transports/file.js:759:10)\n    at new File (/app/code/node_modules/winston/lib/winston/transports/file.js:94:28)\n    at Object.<anonymous> (/app/code/install/web.js:45:3)\n    at Module._compile (node:internal/modules/cjs/loader:1256:14)\n    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)\n    at Module.load (node:internal/modules/cjs/loader:1119:32)\n    at Module._load (node:internal/modules/cjs/loader:960:12)\n    at Module.require (node:internal/modules/cjs/loader:1143:19)\n    at require (node:internal/modules/cjs/helpers:119:18)","trace":[{"column":3,"file":"node:fs","function":"Object.mkdirSync","line":1398,"method":"mkdirSync","native":false},{"column":10,"file":"/app/code/node_modules/winston/lib/winston/transports/file.js","function":"File._createLogDirIfNotExist","line":759,"method":"_createLogDirIfNotExist","native":false},{"column":28,"file":"/app/code/node_modules/winston/lib/winston/transports/file.js","function":"new File","line":94,"method":null,"native":false},{"column":3,"file":"/app/code/install/web.js","function":null,"line":45,"method":null,"native":false},{"column":14,"file":"node:internal/modules/cjs/loader","function":"Module._compile","line":1256,"method":"_compile","native":false},{"column":10,"file":"node:internal/modules/cjs/loader","function":"Module._extensions..js","line":1310,"method":".js","native":false},{"column":32,"file":"node:internal/modules/cjs/loader","function":"Module.load","line":1119,"method":"load","native":false},{"column":12,"file":"node:internal/modules/cjs/loader","function":"Module._load","line":960,"method":"_load","native":false},{"column":19,"file":"node:internal/modules/cjs/loader","function":"Module.require","line":1143,"method":"require","native":false},{"column":18,"file":"node:internal/modules/cjs/helpers","function":"require","line":119,"method":null,"native":false}]}

[Axtremus]

I am observing similar issue after upgrading to NodeBB package version 2.14.x.
Among error logs I am seeing are "ERR!" logs about not being able to resolve the nodebb-plugin-emoji plugins, and the application being "not responding."

Had to downgrade to NodeBB package version 2.13.4 to resolve the error and get the application back up and running.

Funny thing is, if I "upgrade" only to package version 2.14.0, I get only "WARN" logs about not being able to resolve the nodebb-plugin-emoji plugins, and the application will eventually get up and run. But if I take one more step to upgrade to 2.14.1, then the "ERR!" logs come back and the application will be stuck "not responding" again. Downgrading from 2.14.1 back to 2.14.0 did not help -- I get the same "ERR!" logs as mentioned above and the application remains "not responding." I have to downgrade back to 2.13.4 to get out of this hole.

For now, I will disable auto-update and hope that smarter people figure this out and put out a revised NodeBB 2.14.x package that won't have the same or similar issue. (Not really sure how this works ... if 2.14.0 and 2.14.1 really have issues, I'm hoping they will be removed such that auto-update will skip over them and move on to a working update when one becomes available.)

[joseph]

Can you run /app/pkg/start.sh in the web terminal once and then do the nodebb reset ? The reason is that /app/pkg/start.sh will "initialize" the app with necessary run time configuration and directories.

[hpz24]

@joseph Hi, facing similar problems and tried your solution but it does not fix the issue.

[nebulon]

Looks like a dependency issue between plugins. Have you tried to disable some of the mentioned plugins? Maybe they need to be updated upstream and will start working in some time then.

See https://docs.cloudron.io/apps/nodebb/#disabling-plugins how to do this. Otherwise restore the app and only update once those plugins have a new fixed release.

[hpz24]

Disabling the plugins has not been successful: The main problem is, that the terminal stops working when nodebb crashes.
So, I fiddled around a bit more:
I´ve the same problem as @Axtremus:
I downgraded to version 2.13.4 and the app works. Then I updated to 2.14.0 and had also the WARN entries but the app works again. Then I restarted the app manually and the behaviour was the same as at the beginning: WARN entries changed to ERR entries and nothing works.
That means: The problem is in Version 2.14: The container works well till the next restart, then it breaks. Even if the app has daily backups, the backups are useless because the backuped a failed state.
Conclusion: I restored back to 2.13.4, disabled Auto-Update and have lost 9 days of posts and threads ..

[joseph]

@hpz24 said in Issue Starting NodeBB on Cloudron After Server Restart:

The main problem is, that the terminal stops working when nodebb crashes.

Right, so you need to use the recovery mode for this - https://docs.cloudron.io/apps/#recovery-mode . Put app in recovery mode and then if you run /app/pkg/start.sh it will set up the run time directories. After that, you can run nodebb cli and disable plugins.

(cc @girish , please put above in docs)

[Axtremus]

Newbie questions here ...

1. How do I file ticket for this issue?
2. How/what do I monitor to know when this issue is resolved (such that I can reenable Auto-Update again)?

[joseph]

@Axtremus Unfortunately, there is no real way to support issues related to plugins. AFAIK, neither upstream nor Cloudron team tests all the combinations of various plugins and plugin versions and nodebb versions . IOW, one has to debug and find out how to resolve this.

I suggest something like this:

• Take a backup
• Clone the forum - https://docs.cloudron.io/backups/#clone-app
• Experiment with the clone to figure out what the issue is wrt deps. If this is getting too complex, I simply recommend not using those plugins.

At the end of the day, plugins is "code" just dropped into the main code codebase and anything can happen. It's not some simple configuration . For this forum which is nodebb, we barely use any plugins to prevent breakage.

[julian]

Hi, Julian from NodeBB team here. Sorry to hear there are dependency issues, they're tricky to debug.

Can I get a full record of the WARN and ERR messages? From what I can see in the image above, both the emoji-android plugin and reactions plugin have already been updated so that they request the correct version as a peer dependency.