Leantime sessions-store growing fast and endlessly

[msbt]

Hi there!
I just realized that one of my servers is running rather low on disk space even though not a lot new things were added. Cloudrons system page said that docker-volumes is using ~70GB space. After a bit of du'ing I found this volume:

54G     /var/lib/docker/volumes/volumeid/_data/leantime/sessions
54G     /var/lib/docker/volumes/volumeid/_data/leantime
54G     /var/lib/docker/volumes/volumeid/_data
54G     /var/lib/docker/volumes/volumeid

After checking the same inside the apps container I got pointed to this directory /run/leantime/sessions - which seems to be created by https://git.cloudron.io/cloudron/leantime-app/-/blob/main/Dockerfile?ref_type=heads#L38

There are currently ~400k session files in there and counting, each one being 140kb+. It seems to me, that every healthcheck creates a new session file. This is probably a PHP issue, how can we fix that?

[nebulon]

Maybe the app package misses some session cleanup cron job. Do you know leantime better, if this is the case maybe?

[msbt]

@nebulon I'm afraid I do not, a customer of mine is using it with a handful of users (<10). Just checked, the oldest entry in there is from 2023-12-08

[leantime]

Hey, Leantime maintainer here. Thought it might be a good time to chime in.
Leantime doesn't do anything special for session handling. So it appears that this would be related to the php config. More than likely it is the combination of.

session.gc_lifetime
session.gc_divisor
session.gc_probability

https://www.php.net/manual/en/session.configuration.php

[msbt]

@leantime thanks for chiming in!

As suspected, after looking at the timestamps, this has to be the healthcheck:

The files from 08.12.2023 are still there, so it seems there is no limit (or at least not reached yet), after 426k files. Any suggestion what I should try @nebulon? Or a different kind of healthcheck which doesn't produce a session file on every ping?

[nebulon]

@leantime thanks for joining in here and good observation about the healthcheck. @leantime is there a better way to do this instead of just testingg for a 200 status code on / ?

Currently running through the leantime v3.0.0 tests, will see what can be done about session lifetime, but yes ideally they shouldn't be created in the first place.

[nebulon]

The latest package has a session timeout set of 1 week now, so this should remedy the situation at least partly.

[fbartels]

This problem still seems to exist, in addition to the state files being created in /run/leantime/sessions I also see these warnings in /app/code/logs/error.log:

[17-Mar-2024 10:45:28 UTC] PHP Warning:  session_start(): Failed to read session data: files (path: /var/lib/php/sessions) in /app/code/app/Core/Session.php on line 86
[17-Mar-2024 11:00:28 UTC] PHP Warning:  session_start(): open(/var/lib/php/sessions/sess_faca93109effa1af1edd88e4f44a6907151835e9-5a1a3dac769201a6b925ba01be7303f4dc33a595, O_RDWR) failed: Read-only file system (30) in /app/code/app/Core/Session.php on line 86

When looking at /etc/php/8.1/apache2/php.ini I do find session.save_path = /run/leantime/sessions, so it is strange that it still tries /var/lib/php/sessions/ for that.

[nebulon]

Looks like upstream just uses symfony sessions, which in turn should use the php settings as mentioned in https://symfony.com/doc/current/session.html#configuring-garbage-collection

Does anyone here have more experience with the symfony framework?

[fbartels]

I did not yet review the whole source of the app, but the session messages seemingly stopped after I added the session.save_path in /app/data/php.ini and restarted the app. Maybe the wrong php.ini is used by default?

[msbt]

@fbartels did you get that sorted and does it work for you? I have to still have troubles with it and I need to manually check every few weeks which docker container is using up the space and manually remove old sessions:

35G /var/lib/docker/volumes/id/_data/leantime/sessions

my php.ini has the following content:

session.save_path = /run/leantime/sessions
session.gc_probability = 1
session.gc_divisor = 100
session.gc_maxlifetime = 1440

[girish]

@msbt not a real solution, but a hack is to clean out /run/leantime/sessions with the cron job maybe ? https://docs.cloudron.io/apps/#cron

[fbartels]

@msbt said in Leantime sessions-store growing fast and endlessly:

did you get that sorted and does it work for you?

I am not using it myself, but one of my customers does on their Cloudron. since the customer was happy with the workaround and did not complain about further issues I did not make any further research on this.