"Platform data" and "Other" disk usage
-
@yusf: Can you post your solution? Where is the log file located and what did you do with it (just delete it)? What about the "Other"?
-
I just noticed mine at 10.24 GB for "Other". What is that and how do we clear it up (if it's even safe to do so)?
Side note: Would be great if the feature graph would also do the math to show how much remaining space exists so we don't have to do the math ourselves. I think I'll file that feature request separately later on.
-
@d19dotca Please check the size of
/home/yellowtent/platformdata/logs
. We have a logrotate issue which is not cleaning up logs properly. You can delete some of the old files there safely. -
I have the same problem.
I identified:
47G /var/lib/docker
30G /var/backups
Backups i can understand but Docker folder is kind of big.
Any ideea?
du -sh /home/yellowtent/platformdata/*
4.0K /home/yellowtent/platformdata/INFRA_VERSION
4.0K /home/yellowtent/platformdata/VERSION
4.0K /home/yellowtent/platformdata/acme
28K /home/yellowtent/platformdata/addons
12K /home/yellowtent/platformdata/backup
24K /home/yellowtent/platformdata/collectd
67M /home/yellowtent/platformdata/graphite
12K /home/yellowtent/platformdata/logrotate.d
344K /home/yellowtent/platformdata/logs
511M /home/yellowtent/platformdata/mongodb
132M /home/yellowtent/platformdata/mysql
60K /home/yellowtent/platformdata/nginx
77M /home/yellowtent/platformdata/postgresql
1.9M /home/yellowtent/platformdata/redis
4.0K /home/yellowtent/platformdata/sessions
108K /home/yellowtent/platformdata/update -
@Warez Did you use
du -hcs
to get docker storage size? Because the du tool does not work correctly with the overlayfs that docker uses. Instead usedocker system df
output. -
@Warez The 2.5GB there seems OK. Our docker app images are not disk usage optimized.
-
"Other" might not be the best term in hindsight. Basically it is just the sum of everything else on the system. For example the Linux system/distro data itself, as well as data which is otherwise placed outside of Cloudron known folders, if any.
If anyone has a better term for that, to convey this information, I am happy to adjust it, if we find a good one.
-
Hi,
I have another question regardiing this. The following is the output of docker system df.
$ docker system df TYPE TOTAL ACTIVE SIZE RECLAIMABLE Images 34 16 9.857GB 6.508GB (66%) Containers 21 18 0B 0B Local Volumes 160 36 5.274GB 3.964GB (75%) Build Cache 0 0 0B 0B
As you can see, docker claims that a lot of storage is reclaimable.
Is it safe to run
docker volume prune
anddocker image prune
? -
The prune commands should be safe to run. Unless there is some Cloudron bug, it shouldn't remove anything.
-
@girish said in "Platform data" and "Other" disk usage:
We have a logrotate issue which is not cleaning up logs properly.
I still need to continuously watch these log files not filling up the machine completely. Can you take another look at the log rotation feature?
-
@yusf Do you know which log is taking up space? Maybe we miss a logrotation rule somewhere. You can find this by
du -hcs /home/yellowtent/platformdata/logs/*
-
@girish For what it's worth, I ran that command and found the total size was 872 MB. Is that expected? It seems like a lot to me to be nearly a GB in size just for text files. I'd expect that to be more around the 200 MB mark or so, but I'm not sure of the logrotate rules and how much gets logged generally nor how quickly.
I noticed one of the larger log files was around 47 MB on its own, and it looks to be from my Radicale app. It seems to be logging in DEBUG mode, which I presume is not intended? I'm guessing for production systems we'd like to keep logs in a normal INFO mode, right?
-
mm, OK. I looked into your own installation - https://paste.cloudron.io/eqixarerox.css (~800MB)
logrotate configs are in
/home/yellowtent/platformdata/logrotate.d
-
Some apps like
2f004a04-e256-426b-9485-e06a31adc2dd
were missing configs. This is because when we added the new configs many releases ago, we didn't go back and re-generate it for all the apps. To fix this, simply enable and disable recovery mode and then the logrotate config will be generated. -
For
turn
, the logrotate was not added when we added it in the previous release. I have fixed this now for next release. -
For all apps/services, we keep up to 2 logs of 10M. This is why most of them are < 20M. So, this is mostly working as expected.
-
For the rest, I see that log files are > 20M. I wonder why... investigating.
-
-
Found out why. If a log file is > 20M at rotation time, then it's never truncated. Looks like this is just how logrotate works.
-
https://paste.cloudron.io/upozogutut.sql
For me it adds up to 699 MBIs there a way to clean up those logs in between?