Discourse memory problem
-
@jdaviescoates said in Increase minimum RAM for NocoDB:
which makes me think this loop is going to continue for that long before it gives up... Is there a 1hr hard coded limit?
there is some backoff in docker that if it keeps crashing, it does an exponential backoff for restarts.
Is this problem sorted out now? Was this related to disk space?
-
Hmz, the same thing just happened when installing Discourse and that's already got a 4GB minimum! Literally haven't even opened it and it's already run out of memory twice, but the graphs don't show it going up over 2GB
I don't think it ever actually started running. Just kept going into Not Responding.
And it's still the same after cranking it up to nearly 8GB... although, when I looked at the logs it was obviously still doing stuff and starting up, so not sure why it was already showing as Not Responding? But with the increased memory it did eventually get to Running.
But now, just to test again, I'm trying to install another instance to see if that runs out of memory before I even open it too...
... but for the last
101520 (and counting) minutes it's been doing this over and over again:==> Pre-compiling assets Mar 14 05:13:35$ /app/code/app/assets/javascripts/node_modules/.bin/ember build Mar 14 05:13:35Node.js heap_size_limit (1481.5) is less than 2048MB. Setting --max-old-space-size=2048. Mar 14 05:13:35fatal: not a git repository (or any of the parent directories): .git Mar 14 05:13:35yarn run v1.22.19 Mar 14 05:13:37Building Mar 14 05:13:37Environment: development Mar 14 05:13:37WARNING: Ember CLI v5.0.0 is not tested against Node v20.11.0. See "https://github.com/ember-cli/ember-cli/blob/master/docs/node-support.md" to find out which version of Node is best to use. Mar 14 05:13:38WARNING: ember-test-selectors: You are using an unsupported ember-cli-babel version. data-test properties are not automatically stripped from your JS code. Mar 14 05:13:38building... Mar 14 05:13:40...[Babel: @embroider/macros > applyPatches] Mar 14 05:13:40...[Babel: discourse-widget-hbs > applyPatches] Mar 14 05:13:40...[Babel: ember-source > applyPatches] Mar 14 05:13:40...[ConfigLoader] Mar 14 05:13:40=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 05:13:43...[Babel: @ember/legacy-built-in-components > applyPatches] Mar 14 05:13:43...[Babel: @ember/test-helpers > applyPatches] Mar 14 05:13:43...[Babel: dialog-holder > applyPatches] Mar 14 05:13:43...[Babel: discourse-common > applyPatches] Mar 14 05:13:43...[Babel: ember-resize-observer-service > applyPatches] Mar 14 05:13:43...[Babel: float-kit > applyPatches] Mar 14 05:13:43...[Babel: pretty-text > applyPatches] Mar 14 05:13:43...[ember-testing.js] Mar 14 05:13:44...[@embroider/compat/app] Mar 14 05:13:44...[Babel: select-kit > applyPatches] Mar 14 05:13:45...[@embroider/webpack] Mar 14 05:13:50=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 05:13:51...[@embroider/webpack] Mar 14 05:13:57...[@embroider/webpack] Mar 14 05:14:00=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 05:14:06Killed Mar 14 05:14:06error Command failed with exit code 137. Mar 14 05:14:06info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. Mar 14 05:14:07==> Changing permissions Mar 14 05:14:07==> Configuring discourse Mar 14 05:14:07==> Creating directories Mar 14 05:14:07==> Creating symlinks for built-in plugins Mar 14 05:14:07==> Migrating database Mar 14 05:14:09fatal: not a git repository (or any of the parent directories): .git Mar 14 05:14:10=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80I think perhaps something is not right with the package. And looking at my notifications about the memory time outs, it seems they were almost exactly every hour... which makes me think this loop is going to continue for that long before it gives up... Is there a 1hr hard coded limit?
Eventually the logs said:
Mar 14 05:31:2212:M 14 Mar 2024 05:31:22.020 * 10 changes in 300 seconds. Saving... Mar 14 05:31:2212:M 14 Mar 2024 05:31:22.020 * Background saving started by pid 37 Mar 14 05:31:2237:C 14 Mar 2024 05:31:22.045 * DB saved on disk Mar 14 05:31:2237:C 14 Mar 2024 05:31:22.046 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB Mar 14 05:31:2212:M 14 Mar 2024 05:31:22.120 * Background saving terminated with successWhich I think is when the app finally went from Starting... to Not responding, but it's still just stuck on going through the above loop over and over again...
I need to go to bed, but I reckon I'll see an out of memory notification around 06:13 after it's done this for an hour and then given up...
...it was a bit sooner, then hourly:
It's still just looping through this:
Mar 14 08:16:09==> Pre-compiling assets Mar 14 08:16:10=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 08:16:10fatal: not a git repository (or any of the parent directories): .git Mar 14 08:16:11Node.js heap_size_limit (1481.5) is less than 2048MB. Setting --max-old-space-size=2048. Mar 14 08:16:11yarn run v1.22.19 Mar 14 08:16:11$ /app/code/app/assets/javascripts/node_modules/.bin/ember build Mar 14 08:16:12WARNING: Ember CLI v5.0.0 is not tested against Node v20.11.0. See "https://github.com/ember-cli/ember-cli/blob/master/docs/node-support.md" to find out which version of Node is best to use. Mar 14 08:16:12Building Mar 14 08:16:12Environment: development Mar 14 08:16:13WARNING: ember-test-selectors: You are using an unsupported ember-cli-babel version. data-test properties are not automatically stripped from your JS code. Mar 14 08:16:14building... Mar 14 08:16:15...[ConfigLoader] Mar 14 08:16:15...[Babel: @embroider/macros > applyPatches] Mar 14 08:16:16...[Babel: ember-source > applyPatches] Mar 14 08:16:18...[Babel: @glimmer/component > applyPatches] Mar 14 08:16:18...[Babel: pretty-text > applyPatches] Mar 14 08:16:19...[Babel: discourse-common > applyPatches] Mar 14 08:16:19...[Babel: @ember/legacy-built-in-components > applyPatches] Mar 14 08:16:19...[Babel: @ember/test-helpers > applyPatches] Mar 14 08:16:19...[Babel: dialog-holder > applyPatches] Mar 14 08:16:19...[Babel: ember-on-resize-modifier > applyPatches] Mar 14 08:16:19...[Babel: float-kit > applyPatches] Mar 14 08:16:19...[Babel: select-kit > applyPatches] Mar 14 08:16:20=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 08:16:20...[@embroider/compat/app] Mar 14 08:16:21...[@embroider/webpack] Mar 14 08:16:27...[@embroider/webpack] Mar 14 08:16:30=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 08:16:33...[@embroider/webpack] Mar 14 08:16:40=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 08:16:50=> Healtheck error: Error: connect ECONNREFUSED 172.18.17.195:80 Mar 14 08:16:55Killed Mar 14 08:16:55error Command failed with exit code 137. Mar 14 08:16:55info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. Mar 14 08:16:56==> Creating directories Mar 14 08:16:56==> Configuring discourse Mar 14 08:16:56==> Creating symlinks for built-in plugins Mar 14 08:16:56==> Changing permissions Mar 14 08:16:56==> Migrating database Mar 14 08:16:5712:M 14 Mar 2024 08:16:57.060 * 10 changes in 300 seconds. Saving... Mar 14 08:16:5712:M 14 Mar 2024 08:16:57.060 * Background saving started by pid 70 Mar 14 08:16:5770:C 14 Mar 2024 08:16:57.087 * DB saved on disk Mar 14 08:16:5770:C 14 Mar 2024 08:16:57.088 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB Mar 14 08:16:5712:M 14 Mar 2024 08:16:57.161 * Background saving terminated with success Mar 14 08:16:57fatal: not a git repository (or any of the parent directories): .git -
@girish said in Increase minimum RAM for NocoDB:
Was this related to disk space?
I don't think so as this repeat test was done after sorting out my disk space issue.
It's still looping through the same thing and running out of memory on the hour.
I've just bumped the memory up to 4.25 to see what happens...
-
@jdaviescoates said in Increase minimum RAM for NocoDB:
I've just bumped the memory up to 4.25 to see what happens...
Didn't make any difference... bumped it to 4.5...
-
Tested on the demo (although that's on 7.7.0 whereas I'm still on 7.6.4) and Discourse installed fine with 4GB RAM.
But on my system it just will not install with 4GB, the same thing happens every time. Makes me wonder if there might be some errors in my RAM or something (any ideas how I'd check that?)
But If I crank the memory up, it installs: I just tried yet another test, failed exactly as above. Then cranked it up to 6.25 and now it's running...
-
Tested on the demo (although that's on 7.7.0 whereas I'm still on 7.6.4) and Discourse installed fine with 4GB RAM.
But on my system it just will not install with 4GB, the same thing happens every time. Makes me wonder if there might be some errors in my RAM or something (any ideas how I'd check that?)
But If I crank the memory up, it installs: I just tried yet another test, failed exactly as above. Then cranked it up to 6.25 and now it's running...
Just tested with 5.13 GB too and that also worked...
On my system Discourse fails to instal with 4.88 GB, but with 5.13 GB it works (the existing slider makes it basically impossible for me to test anything between these figures)
-
@jdaviescoates said in Increase minimum RAM for NocoDB:
Makes me wonder if there might be some errors in my RAM or something (any ideas how I'd check that?)
I installed memtester on the server and tested 4GB 4 times and it all came out OK:
memtester 4G 4 memtester version 4.5.1 (64-bit) Copyright (C) 2001-2020 Charles Cazabon. Licensed under the GNU General Public License version 2 (only). pagesize is 4096 pagesizemask is 0xfffffffffffff000 want 4096MB (4294967296 bytes) got 4096MB (4294967296 bytes), trying mlock ...locked. Loop 1/4: Stuck Address : ok Random Value : ok Compare XOR : ok Compare SUB : ok Compare MUL : ok Compare DIV : ok Compare OR : ok Compare AND : ok Sequential Increment: ok Solid Bits : ok Block Sequential : ok Checkerboard : ok Bit Spread : ok Bit Flip : ok Walking Ones : ok Walking Zeroes : ok 8-bit Writes : ok 16-bit Writes : ok Loop 2/4: Stuck Address : ok Random Value : ok Compare XOR : ok Compare SUB : ok Compare MUL : ok Compare DIV : ok Compare OR : ok Compare AND : ok Sequential Increment: ok Solid Bits : ok Block Sequential : ok Checkerboard : ok Bit Spread : ok Bit Flip : ok Walking Ones : ok Walking Zeroes : ok 8-bit Writes : ok 16-bit Writes : ok Loop 3/4: Stuck Address : ok Random Value : ok Compare XOR : ok Compare SUB : ok Compare MUL : ok Compare DIV : ok Compare OR : ok Compare AND : ok Sequential Increment: ok Solid Bits : ok Block Sequential : ok Checkerboard : ok Bit Spread : ok Bit Flip : ok Walking Ones : ok Walking Zeroes : ok 8-bit Writes : ok 16-bit Writes : ok Loop 4/4: Stuck Address : ok Random Value : ok Compare XOR : ok Compare SUB : ok Compare MUL : ok Compare DIV : ok Compare OR : ok Compare AND : ok Sequential Increment: ok Solid Bits : ok Block Sequential : ok Checkerboard : ok Bit Spread : ok Bit Flip : ok Walking Ones : ok Walking Zeroes : ok 8-bit Writes : ok 16-bit Writes : ok -
The Discourse app that eventually installed when I upped the memory to 5.13 GB RAM, but which I've never even opened, just ran out of RAM
which just doesn't seem right.(@staff I guess perhaps we should move all these posts from #5 onwards into the Discourse category?)
