v3.24.0 constantly runs out of memory

[shrey]

Yeah, same here. Had to roll back to the last version.

[ccfu]

Had this as well several times after uodate, but it seems to have stabilised. No OOM for the past 8 hours.

[timconsidine]

Cloudron Team : I am not clear whether cloudron 3.24.0 used a different upstream base 3.1.0 to previous, but my impression is not. So the issue is in cloudron v3.24.0 ? Can it be fixed ?

[james]

Hello @timconsidine
These are the changes made https://git.cloudron.io/packages/umami-app/-/commit/4e3d31c2f37285960d017fd157956b5aae9138ef

One thing that jumps at me is --max-old-space-size=4096.

The container runs a full Next.js production yarn build on every start (see start.sh:L35).
Next.js builds are notoriously memory-hungry; they scale with the size of the app bundle.
Node is told --max-old-space-size=4096 (4 GiB V8 heap) while the cgroup hard cap from CloudronManifest.json:L13 is 2 GiB by default.
Node happily tries to grow past 2 GiB, the kernel OOM-kills the container.

This value should not be hard coded, instead should reflect the memory limit set by the user.
An app update will be available soon.

[timconsidine]

@james thank you, I appreciate you looking into and resolving it

[james]

Fixed with the latest update https://forum.cloudron.io/post/123645

[ekevu123]

I have just run into the same issue.

Isn't the issue that the build process forks into different processes per worker and the sum of it takes too much memory? I had to set my memory limit to 5 GB to get the app to start again. It looks to me like the workers are given 75 % of the memory assigned to umami each, which leads to the issue. Also, does the app have to recompile every single time it restarts?
Maybe the default memory could also be raised above 2 GB considering that several people (also in another thread) faced memory issues before.

[d19dotca]

I too have just run into this with the most recent package/image from Cloudron for Umami... it was constantly hitting OOM and failing to respond until I lifted it from 2 GB to 5 GB temporarily and then could reduce it back down again. I think the issue was that it temporarily needed much more than 2 GB of memory to perform what appeared to be some database updates for Umami.

This was for package 3.27.0, which contains Umami 3.4.0.

[timconsidine]

@d19dotca FWIW I have switched from Umami to Rybbit (Community App).

Umami performs well, I just prefer Rybbit.

[d19dotca]

@timconsidine Yeah I think I prefer Rybbit too, right now I haven’t switched over yet though and am running both in parallel, with Rybbit taking a few sites too.