Disk usage is not showing

[roru2k20]

@girish said in Disk usage is not showing:

we only scan ext4 disks

I think that is the reason for both problem. I use a cifs Share and @necrevistonnezr use a ext2 Partition Format.

I try temporarily to unmount my backup drive and look into the view.

[roru2k20]

Short Test while I have umount my backup drive to look in my disk usage.
Unfortunately no change.

[necrevistonnezr]

@roru2k20 said in Disk usage is not showing:

@girish said in Disk usage is not showing:

we only scan ext4 disks

I think that is the reason for both problem. I use a cifs Share and @necrevistonnezr use a ext2 Partition Format.

I try temporarily to unmount my backup drive and look into the view.

But only /boot is formatted with ext2?! (and I don’t really know why...)

[lolliop]

I'm having a similar issue as well.
Edit: Please let me know if I need to open a new thread.
Edit2: Not a new system. It's been running for about a week.

Vultr, Ubuntu 20.04.1, Cloudron 5.6.3

System Info

Chrome - Console log

df -Th

Filesystem     Type      Size  Used Avail Use% Mounted on
udev           devtmpfs  1.9G     0  1.9G   0% /dev
tmpfs          tmpfs     394M  1.7M  392M   1% /run
/dev/vda1      ext4       76G   15G   57G  21% /
tmpfs          tmpfs     2.0G     0  2.0G   0% /dev/shm
tmpfs          tmpfs     5.0M     0  5.0M   0% /run/lock
tmpfs          tmpfs     2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs          tmpfs     394M   24K  394M   1% /run/user/125
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/4aa0476d5e9f54a77fb245e7c391ce49bc6a80fc7120300cc8933261bd32878a/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/a6ea14043f42c07087d1f133d33d50e161cef9342b8360ed6601c3bdc292dfcb/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/1b24ff91fc4450d69d8419879fc46925d20327bfcafdb466e143752a7b5db1a1/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/60e4893426213b77982f2b47efca482389d720e7148ed9a4cb205348e8929ca6/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/69b4dd2921b7120b9d3b33089d9cbe945655bdddd61df8f5d06ece86c461350c/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/dea7c24f3096cb7737d60b10794ce8c2d3d4249f0c48707f54d6a3fa443508ee/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/c28552a10f0f0736d46f6be64046b05a5195fd16b9f844a9fd95316ecf3755e5/merged
overlay        overlay    76G   15G   57G  21% /var/lib/docker/overlay2/0d8cf0c75789aab0bfc2a2802995a65c06e7a0476ba4315ff73ece45d34e25e0/merged
tmpfs          tmpfs     394M     0  394M   0% /run/user/0

Cloudron system log

Dec 02 11:59:50 box:apphealthmonitor app health: 1 alive / 0 dead.
Dec 02 12:00:00 box:disks Checking disk space
Dec 02 12:00:00 box:janitor Cleaning up expired tokens
Dec 02 12:00:00 box:janitor Cleaned up 0 expired tokens.
Dec 02 12:00:00 box:tasks startTask - starting task 40. logs at /home/yellowtent/platformdata/logs/tasks/40.log
Dec 02 12:00:00 box:shell startTask spawn: /usr/bin/sudo -S -E /home/yellowtent/box/src/scripts/starttask.sh 40 /home/yellowtent/platformdata/logs/tasks/40.log 0 400
Dec 02 12:00:00 box:disks checkDiskSpace: disk space checked. ok: true
Dec 02 12:00:00 box:notifications alert: id=diskSpace title=Server is running out of disk space
Dec 02 12:00:00 box:apphealthmonitor app health: 1 alive / 0 dead.
Dec 02 12:00:00 box:shell startTask (stdout): Running as unit: box-task-40.service
Dec 02 12:00:01 box:shell startTask (stdout): Finished with result: success
Dec 02 12:00:01 box:shell startTask (stdout): Main processes terminated with: code=exited/status=0
Dec 02 12:00:01 box:shell startTask (stdout): Service runtime: 1.164s
Dec 02 12:00:01 box:shell startTask (stdout): Service box-task-40 finished with exit code 0
Dec 02 12:00:01 box:tasks startTask: 40 completed with code 0 and signal 0
Dec 02 12:00:01 box:tasks startTask: 40 done
Dec 02 12:00:10 box:apphealthmonitor app health: 1 alive / 0 dead.

[msbt]

@lolliop if you're actually on Ubuntu 20 - which is not supported just yet - that might be the reason

[mehdi]

@lolliop Yes, as @msbt just said, you should not be on Ubuntu 20.04. I'm surprised, how did you end up with cloudron on ubuntu 20 ?
Did you install it directly on 20.04, and the installer worked ? (It should not)
Or did you update the underlying Ubuntu while Cloudron was already installed ? (In that case, I am very surprised that this disk usage thing is the only thing that broke ^^)

[girish]

I suspect @lolliop did a dist upgrade to Ubuntu 20. This would have broken graphite/collected which is what we use for graphs since the collected/python3 plugins don't work on ubuntu 20 out of the box. See https://bugs.launchpad.net/ubuntu/+source/collectd/+bug/1872281 and https://bugs.archlinux.org/task/64533 . @lolliop can you confirm?

[lolliop]

Oh. My mistake then.

I thought Ubuntu 20.04 is already supported, because I checked the script, cloudron-setup, and it no longer prohibits me from installing on Ubuntu Fossa (It used to though).

I didn't do a dist upgrade. It was installed on a clean Ubuntu 20.04.1.

So far, everything else seems to be OK (touch wood). But then I'm just testing Open Project.

[girish]

@lolliop You can try this:

echo -e "\nLD_PRELOAD=/usr/lib/python3.8/config-3.8-x86_64-linux-gnu/libpython3.8.so" >> /etc/default/collectd

then systemctl restart collectd. And graphs might appear in some time.

As for Ubuntu 20.04 support, indeed we are just releasing Cloudron 6 which will support Focal. So, you caught us in between the release process.

[necrevistonnezr]

@necrevistonnezr said in Disk usage is not showing:

@roru2k20 said in Disk usage is not showing:

@girish said in Disk usage is not showing:

we only scan ext4 disks

I think that is the reason for both problem. I use a cifs Share and @necrevistonnezr use a ext2 Partition Format.

I try temporarily to unmount my backup drive and look into the view.

But only /boot is formatted with ext2?! (and I don’t really know why...)

Mine are still not showing under 18.04...

[lolliop]

@girish,

No dice. Rebooted and waited for like 3 hours.
I'll update if it changes later.

It's ok. I'll just wait for 6.0.
Thank you.

[girish]

@lolliop @necrevistonnezr I think I found the issue for Ubuntu 20 atleast. Can you please try https://git.cloudron.io/cloudron/box/-/commit/6810d823f542808860955196d959fadaba0a9ab8 ? The file is at /home/yellowtent/box/setup/start/collectd/df.py. After you make the change systemctl restart collectd. You can check the logs in /home/yellowtent/platfordata/logs/collectd/app.log.

[lolliop]

@girish,

Yay!!! It's working now.
But I think the logs are at /home/yellowtent/platfordata/logs/collectd/collectd.log.

Thank you!

[necrevistonnezr]

@girish said in Disk usage is not showing:

@lolliop @necrevistonnezr I think I found the issue for Ubuntu 20 atleast. Can you please try https://git.cloudron.io/cloudron/box/-/commit/6810d823f542808860955196d959fadaba0a9ab8 ? The file is at /home/yellowtent/box/setup/start/collectd/df.py. After you make the change systemctl restart collectd. You can check the logs in /home/yellowtent/platfordata/logs/collectd/app.log.

I still don't see any data after 10 minutes. The log after the change you mentioned (mind you, I'm still on 18.04):

[2020-12-07 14:06:27] Exiting normally.
[2020-12-07 14:06:27] collectd: Stopping 5 read threads.
[2020-12-07 14:06:27]   error getting the size of /media/SG2TB/CloudronBackup: list index out of range
[2020-12-07 14:06:27] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data"
[2020-12-07 14:06:27]   size of /home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data is 18436096 (time: 1607349987)
[2020-12-07 14:06:33] size of docker images is 12.56GB (12560000000) (time: 1607349993)
[2020-12-07 14:06:33] collectd: Stopping 5 write threads.
[2020-12-07 14:06:33] plugin_load: plugin "aggregation" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "cpu" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "interface" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "load" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "memory" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "python" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "swap" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "write_graphite" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata/mail
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/platformdata
[2020-12-07 14:06:33] plugin_load: plugin "table" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/M/Music
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/CloudronBackup
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data
[2020-12-07 14:06:33] plugin_load: plugin "aggregation" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "cpu" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "interface" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "load" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "memory" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "python" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "swap" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "write_graphite" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata/mail
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/platformdata
[2020-12-07 14:06:33] plugin_load: plugin "table" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/M/Music
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/CloudronBackup
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data
[2020-12-07 14:06:33] Systemd detected, trying to signal readyness.
[2020-12-07 14:06:33] custom du plugin initialized with [{'exclude': '', 'name': 'maildata', 'dir': '/home/yellowtent/boxdata/mail'}, {'exclude': 'mail', 'name': 'boxdata', 'dir': '/home/yellowtent/boxdata'}, {'exclude': '', 'name': 'platformdata', 'dir': '/home/yellowtent/platformdata'}, {'exclude': '', 'name': '1d6c5d10-67aa-4cbe-b21d-20e9b7688d83', 'dir': '/home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data'}, {'exclude': '', 'name': '29e278a4-c079-4401-942c-e0e17f588482', 'dir': '/home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data'}, {'exclude': '', 'name': '4a0b6b44-5cab-4c04-afae-5e078727dbe4', 'dir': '/home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data'}, {'exclude': '', 'name': '6c1ecf49-b8f4-4a77-ae31-56a6c53868cd', 'dir': '/home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data'}, {'exclude': '', 'name': '6d1f25e8-39ca-46e2-b8a7-fb9db45c7a9b', 'dir': '/media/SG2TB/M/Music'}, {'exclude': '', 'name': '91a711c2-51b8-4095-8250-3168e957eb1b', 'dir': '/home/yellowten
[2020-12-07 14:06:34] custom df plugin initialized with [[u'/dev/mapper/beebox--vg-root', u'/', u'476261416', u'421197060', u'30848600'], [u'/dev/sda1', u'/media/SG2TB', u'1922728820', u'1631095852', u'193940908']]
[2020-12-07 14:06:34] Initialization complete, entering read-loop.
[2020-12-07 14:06:34] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/boxdata/mail"
[2020-12-07 14:06:36]   size of /home/yellowtent/boxdata/mail is 6544105472 (time: 1607349996)
[2020-12-07 14:06:36] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/boxdata" --exclude "mail"
[2020-12-07 14:06:36]   size of /home/yellowtent/boxdata is 4628480 (time: 1607349996)
[2020-12-07 14:06:36] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/platformdata"
[2020-12-07 14:06:37]   size of /home/yellowtent/platformdata is 2807963648 (time: 1607349996)
[2020-12-07 14:06:37] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data"
[2020-12-07 14:06:37]   size of /home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data is 34973192192 (time: 1607349996)
[2020-12-07 14:06:37] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data"
[2020-12-07 14:07:11]   size of /home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data is 345786105856 (time: 1607350030)
[2020-12-07 14:07:11] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data"
[2020-12-07 14:07:11]   size of /home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data is 1777664 (time: 1607350030)
[2020-12-07 14:07:11] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data"
[2020-12-07 14:07:11]   size of /home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data is 12288 (time: 1607350030)
[2020-12-07 14:07:11] computing size with command: timeout 1800 du -DsB1 "/media/SG2TB/M/Music"
[2020-12-07 14:07:19]   size of /media/SG2TB/M/Music is 291166932992 (time: 1607350039)
[2020-12-07 14:07:19] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data"
[2020-12-07 14:07:19]   size of /home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data is 49152 (time: 1607350039)
[2020-12-07 14:07:19] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data"
[2020-12-07 14:07:20]   size of /home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data is 12730368 (time: 1607350039)
[2020-12-07 14:07:20] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data"
[2020-12-07 14:07:20]   size of /home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data is 40992768 (time: 1607350039)
[2020-12-07 14:07:20] computing size with command: timeout 1800 du -DsB1 "/media/SG2TB/CloudronBackup"

[necrevistonnezr]

@girish
I‘m still not seeing my disks in 6.1.2.
Now I get the following error:

[nebulon]

Is this error shown consistently? While being broken twice, the first issue is that the graphite service does not respond. Check the services view and possibly increase the memory limit if it doesn't start.

Secondly it clearly shouldn't deliver that error like this

[necrevistonnezr]

@nebulon
Error persists even after increasing Graphite memory to 1GB and making sure it‘s running....

[necrevistonnezr]

@necrevistonnezr said in Disk usage is not showing:

@nebulon
Error persists even after increasing Graphite memory to 1GB and making sure it‘s running....

Mind you that this problem still persists.

BTW when I df -h on my server, I get

Dateisystem                 Größe Benutzt Verf. Verw% Eingehängt auf
udev                         3,7G       0  3,7G    0% /dev
tmpfs                        764M    3,5M  760M    1% /run
/dev/mapper/beebox--vg-root  455G    216G  216G   50% /
tmpfs                        3,8G    4,0K  3,8G    1% /dev/shm
tmpfs                        5,0M    4,0K  5,0M    1% /run/lock
tmpfs                        3,8G       0  3,8G    0% /sys/fs/cgroup
/dev/loop2                   100M    100M     0  100% /snap/core/10859
/dev/loop3                    70M     70M     0  100% /snap/lxd/19188
/dev/loop4                    71M     71M     0  100% /snap/lxd/19647
/dev/loop0                   9,2M    9,2M     0  100% /snap/canonical-livepatch/95
/dev/loop5                    56M     56M     0  100% /snap/core18/1988
/dev/loop6                   9,2M    9,2M     0  100% /snap/canonical-livepatch/94
/dev/loop7                   100M    100M     0  100% /snap/core/10908
/dev/sdb1                    472M    201M  247M   45% /boot
/dev/sda1                    1,8T    1,2T  603G   66% /media/SG2TB
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/555d1b293de54c90f24cdb5606ec8a63e5b7572cb3aa3e01a4cf3eff6c4764a3/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/569210af31ab431e32dbe442ac97cf218302a5de6ecf2cf92b09e3c9275f5e37/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/2e5b4a84fbbf655d9ecf867c89e470ca5327b51cf922a4d587fd70ff3628ab96/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/a629a8c09a8543cd2cea1068b518cd7daa3ee5d9b4e73e9590421a84c88ea5d4/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/ea30d49958268a686338ab1730430159cc3cd2c67e5f54225afc545d98d4cfaf/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/70124b82723555ee8be71a4da7da01986291a4c70c68fb13875d8355e9431b3c/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/308aeca45b92c222a1f8407206188c8d4f557d71523100effbd0f3b5a0ca9d0b/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/80a478da3839d30895bb142769a76a692a2881783bf67a96867cd0d444183b9a/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/c8e2f1ac2b7033a16731e81ce8bb321469ed478a0ef7838d464793cdec1e16f6/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/a8efa5e8d2af284f583c829f6b0fb418a417e09f17b6644a5a3b12de904b5a40/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/e0d4682e13f35d3c6e9ac93695bc6ef937e179ba614c28c06a189a51ed6e0928/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/c5285c38213dc2ac3c096419b407208600e94af8996ab5c9670313e73a3226c8/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/083ea362abeb6ada746d9169f4b600202661997788f9578f8e81c72a402831ec/merged
/dev/loop8                    56M     56M     0  100% /snap/core18/1997
tmpfs                        764M       0  764M    0% /run/user/1000

Is that a correct display? I didn't have these "overlays" before....

[girish]

@necrevistonnezr I recently made this change which fixed it in another user's machine which had de locale ( I see yours is as well ). Can you please try this - https://git.cloudron.io/cloudron/box/-/commit/f7bd47888ac157fc5b86021f6458b6a7f0d3fba8

• Edit /etc/systemd/system/box.service. Add "LC_ALL=C" to the Environment key.
• systemctl daemon-reload
• systemctl restart box

Atleast, this was one part of the problem.

[necrevistonnezr]

@girish Great, that works! I guess that's not permanent until the next Cloudron update, correct?