Disk usage is not showing

[mehdi]

@lolliop Yes, as @msbt just said, you should not be on Ubuntu 20.04. I'm surprised, how did you end up with cloudron on ubuntu 20 ?
Did you install it directly on 20.04, and the installer worked ? (It should not)
Or did you update the underlying Ubuntu while Cloudron was already installed ? (In that case, I am very surprised that this disk usage thing is the only thing that broke ^^)

[girish]

I suspect @lolliop did a dist upgrade to Ubuntu 20. This would have broken graphite/collected which is what we use for graphs since the collected/python3 plugins don't work on ubuntu 20 out of the box. See https://bugs.launchpad.net/ubuntu/+source/collectd/+bug/1872281 and https://bugs.archlinux.org/task/64533 . @lolliop can you confirm?

[lolliop]

Oh. My mistake then.

I thought Ubuntu 20.04 is already supported, because I checked the script, cloudron-setup, and it no longer prohibits me from installing on Ubuntu Fossa (It used to though).

I didn't do a dist upgrade. It was installed on a clean Ubuntu 20.04.1.

So far, everything else seems to be OK (touch wood). But then I'm just testing Open Project.

[girish]

@lolliop You can try this:

echo -e "\nLD_PRELOAD=/usr/lib/python3.8/config-3.8-x86_64-linux-gnu/libpython3.8.so" >> /etc/default/collectd

then systemctl restart collectd. And graphs might appear in some time.

As for Ubuntu 20.04 support, indeed we are just releasing Cloudron 6 which will support Focal. So, you caught us in between the release process.

[necrevistonnezr]

@necrevistonnezr said in Disk usage is not showing:

@roru2k20 said in Disk usage is not showing:

@girish said in Disk usage is not showing:

we only scan ext4 disks

I think that is the reason for both problem. I use a cifs Share and @necrevistonnezr use a ext2 Partition Format.

I try temporarily to unmount my backup drive and look into the view.

But only /boot is formatted with ext2?! (and I don’t really know why...)

Mine are still not showing under 18.04...

[lolliop]

@girish,

No dice. Rebooted and waited for like 3 hours.
I'll update if it changes later.

It's ok. I'll just wait for 6.0.
Thank you.

[girish]

@lolliop @necrevistonnezr I think I found the issue for Ubuntu 20 atleast. Can you please try https://git.cloudron.io/cloudron/box/-/commit/6810d823f542808860955196d959fadaba0a9ab8 ? The file is at /home/yellowtent/box/setup/start/collectd/df.py. After you make the change systemctl restart collectd. You can check the logs in /home/yellowtent/platfordata/logs/collectd/app.log.

[lolliop]

@girish,

Yay!!! It's working now.
But I think the logs are at /home/yellowtent/platfordata/logs/collectd/collectd.log.

Thank you!

[necrevistonnezr]

@girish said in Disk usage is not showing:

@lolliop @necrevistonnezr I think I found the issue for Ubuntu 20 atleast. Can you please try https://git.cloudron.io/cloudron/box/-/commit/6810d823f542808860955196d959fadaba0a9ab8 ? The file is at /home/yellowtent/box/setup/start/collectd/df.py. After you make the change systemctl restart collectd. You can check the logs in /home/yellowtent/platfordata/logs/collectd/app.log.

I still don't see any data after 10 minutes. The log after the change you mentioned (mind you, I'm still on 18.04):

[2020-12-07 14:06:27] Exiting normally.
[2020-12-07 14:06:27] collectd: Stopping 5 read threads.
[2020-12-07 14:06:27]   error getting the size of /media/SG2TB/CloudronBackup: list index out of range
[2020-12-07 14:06:27] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data"
[2020-12-07 14:06:27]   size of /home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data is 18436096 (time: 1607349987)
[2020-12-07 14:06:33] size of docker images is 12.56GB (12560000000) (time: 1607349993)
[2020-12-07 14:06:33] collectd: Stopping 5 write threads.
[2020-12-07 14:06:33] plugin_load: plugin "aggregation" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "cpu" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "interface" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "load" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "memory" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "python" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "swap" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "write_graphite" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata/mail
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/platformdata
[2020-12-07 14:06:33] plugin_load: plugin "table" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/M/Music
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/CloudronBackup
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data
[2020-12-07 14:06:33] plugin_load: plugin "aggregation" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "cpu" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "interface" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "load" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "memory" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "python" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "swap" successfully loaded.
[2020-12-07 14:06:33] plugin_load: plugin "write_graphite" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata/mail
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/boxdata
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/platformdata
[2020-12-07 14:06:33] plugin_load: plugin "table" successfully loaded.
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/M/Music
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data
[2020-12-07 14:06:33] du plugin: monitoring /media/SG2TB/CloudronBackup
[2020-12-07 14:06:33] du plugin: monitoring /home/yellowtent/appsdata/f393a761-8c10-46a6-83e7-486fa129a2d3/data
[2020-12-07 14:06:33] Systemd detected, trying to signal readyness.
[2020-12-07 14:06:33] custom du plugin initialized with [{'exclude': '', 'name': 'maildata', 'dir': '/home/yellowtent/boxdata/mail'}, {'exclude': 'mail', 'name': 'boxdata', 'dir': '/home/yellowtent/boxdata'}, {'exclude': '', 'name': 'platformdata', 'dir': '/home/yellowtent/platformdata'}, {'exclude': '', 'name': '1d6c5d10-67aa-4cbe-b21d-20e9b7688d83', 'dir': '/home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data'}, {'exclude': '', 'name': '29e278a4-c079-4401-942c-e0e17f588482', 'dir': '/home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data'}, {'exclude': '', 'name': '4a0b6b44-5cab-4c04-afae-5e078727dbe4', 'dir': '/home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data'}, {'exclude': '', 'name': '6c1ecf49-b8f4-4a77-ae31-56a6c53868cd', 'dir': '/home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data'}, {'exclude': '', 'name': '6d1f25e8-39ca-46e2-b8a7-fb9db45c7a9b', 'dir': '/media/SG2TB/M/Music'}, {'exclude': '', 'name': '91a711c2-51b8-4095-8250-3168e957eb1b', 'dir': '/home/yellowten
[2020-12-07 14:06:34] custom df plugin initialized with [[u'/dev/mapper/beebox--vg-root', u'/', u'476261416', u'421197060', u'30848600'], [u'/dev/sda1', u'/media/SG2TB', u'1922728820', u'1631095852', u'193940908']]
[2020-12-07 14:06:34] Initialization complete, entering read-loop.
[2020-12-07 14:06:34] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/boxdata/mail"
[2020-12-07 14:06:36]   size of /home/yellowtent/boxdata/mail is 6544105472 (time: 1607349996)
[2020-12-07 14:06:36] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/boxdata" --exclude "mail"
[2020-12-07 14:06:36]   size of /home/yellowtent/boxdata is 4628480 (time: 1607349996)
[2020-12-07 14:06:36] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/platformdata"
[2020-12-07 14:06:37]   size of /home/yellowtent/platformdata is 2807963648 (time: 1607349996)
[2020-12-07 14:06:37] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data"
[2020-12-07 14:06:37]   size of /home/yellowtent/appsdata/1d6c5d10-67aa-4cbe-b21d-20e9b7688d83/data is 34973192192 (time: 1607349996)
[2020-12-07 14:06:37] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data"
[2020-12-07 14:07:11]   size of /home/yellowtent/appsdata/29e278a4-c079-4401-942c-e0e17f588482/data is 345786105856 (time: 1607350030)
[2020-12-07 14:07:11] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data"
[2020-12-07 14:07:11]   size of /home/yellowtent/appsdata/4a0b6b44-5cab-4c04-afae-5e078727dbe4/data is 1777664 (time: 1607350030)
[2020-12-07 14:07:11] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data"
[2020-12-07 14:07:11]   size of /home/yellowtent/appsdata/6c1ecf49-b8f4-4a77-ae31-56a6c53868cd/data is 12288 (time: 1607350030)
[2020-12-07 14:07:11] computing size with command: timeout 1800 du -DsB1 "/media/SG2TB/M/Music"
[2020-12-07 14:07:19]   size of /media/SG2TB/M/Music is 291166932992 (time: 1607350039)
[2020-12-07 14:07:19] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data"
[2020-12-07 14:07:19]   size of /home/yellowtent/appsdata/91a711c2-51b8-4095-8250-3168e957eb1b/data is 49152 (time: 1607350039)
[2020-12-07 14:07:19] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data"
[2020-12-07 14:07:20]   size of /home/yellowtent/appsdata/b7ace111-fdd3-484f-bc6c-f8ad0b2e811f/data is 12730368 (time: 1607350039)
[2020-12-07 14:07:20] computing size with command: timeout 1800 du -DsB1 "/home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data"
[2020-12-07 14:07:20]   size of /home/yellowtent/appsdata/c5c137fa-0221-4d42-b418-10e544cebad1/data is 40992768 (time: 1607350039)
[2020-12-07 14:07:20] computing size with command: timeout 1800 du -DsB1 "/media/SG2TB/CloudronBackup"

[necrevistonnezr]

@girish
I‘m still not seeing my disks in 6.1.2.
Now I get the following error:

[nebulon]

Is this error shown consistently? While being broken twice, the first issue is that the graphite service does not respond. Check the services view and possibly increase the memory limit if it doesn't start.

Secondly it clearly shouldn't deliver that error like this

[necrevistonnezr]

@nebulon
Error persists even after increasing Graphite memory to 1GB and making sure it‘s running....

[necrevistonnezr]

@necrevistonnezr said in Disk usage is not showing:

@nebulon
Error persists even after increasing Graphite memory to 1GB and making sure it‘s running....

Mind you that this problem still persists.

BTW when I df -h on my server, I get

Dateisystem                 Größe Benutzt Verf. Verw% Eingehängt auf
udev                         3,7G       0  3,7G    0% /dev
tmpfs                        764M    3,5M  760M    1% /run
/dev/mapper/beebox--vg-root  455G    216G  216G   50% /
tmpfs                        3,8G    4,0K  3,8G    1% /dev/shm
tmpfs                        5,0M    4,0K  5,0M    1% /run/lock
tmpfs                        3,8G       0  3,8G    0% /sys/fs/cgroup
/dev/loop2                   100M    100M     0  100% /snap/core/10859
/dev/loop3                    70M     70M     0  100% /snap/lxd/19188
/dev/loop4                    71M     71M     0  100% /snap/lxd/19647
/dev/loop0                   9,2M    9,2M     0  100% /snap/canonical-livepatch/95
/dev/loop5                    56M     56M     0  100% /snap/core18/1988
/dev/loop6                   9,2M    9,2M     0  100% /snap/canonical-livepatch/94
/dev/loop7                   100M    100M     0  100% /snap/core/10908
/dev/sdb1                    472M    201M  247M   45% /boot
/dev/sda1                    1,8T    1,2T  603G   66% /media/SG2TB
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/555d1b293de54c90f24cdb5606ec8a63e5b7572cb3aa3e01a4cf3eff6c4764a3/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/569210af31ab431e32dbe442ac97cf218302a5de6ecf2cf92b09e3c9275f5e37/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/2e5b4a84fbbf655d9ecf867c89e470ca5327b51cf922a4d587fd70ff3628ab96/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/a629a8c09a8543cd2cea1068b518cd7daa3ee5d9b4e73e9590421a84c88ea5d4/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/ea30d49958268a686338ab1730430159cc3cd2c67e5f54225afc545d98d4cfaf/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/70124b82723555ee8be71a4da7da01986291a4c70c68fb13875d8355e9431b3c/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/308aeca45b92c222a1f8407206188c8d4f557d71523100effbd0f3b5a0ca9d0b/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/80a478da3839d30895bb142769a76a692a2881783bf67a96867cd0d444183b9a/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/c8e2f1ac2b7033a16731e81ce8bb321469ed478a0ef7838d464793cdec1e16f6/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/a8efa5e8d2af284f583c829f6b0fb418a417e09f17b6644a5a3b12de904b5a40/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/e0d4682e13f35d3c6e9ac93695bc6ef937e179ba614c28c06a189a51ed6e0928/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/c5285c38213dc2ac3c096419b407208600e94af8996ab5c9670313e73a3226c8/merged
overlay                      455G    216G  216G   50% /var/lib/docker/overlay2/083ea362abeb6ada746d9169f4b600202661997788f9578f8e81c72a402831ec/merged
/dev/loop8                    56M     56M     0  100% /snap/core18/1997
tmpfs                        764M       0  764M    0% /run/user/1000

Is that a correct display? I didn't have these "overlays" before....

[girish]

@necrevistonnezr I recently made this change which fixed it in another user's machine which had de locale ( I see yours is as well ). Can you please try this - https://git.cloudron.io/cloudron/box/-/commit/f7bd47888ac157fc5b86021f6458b6a7f0d3fba8

• Edit /etc/systemd/system/box.service. Add "LC_ALL=C" to the Environment key.
• systemctl daemon-reload
• systemctl restart box

Atleast, this was one part of the problem.

[necrevistonnezr]

@girish Great, that works! I guess that's not permanent until the next Cloudron update, correct?

[nebulon]

@necrevistonnezr the next update will have that fixed and until that the change will persist also restarts.

[taisau]

My Disk Usage panel only shows a spinner. I think this started after updating to v7.6.3 today.

[girish]

@taisau do you have a cifs or some other network mounted disk connected?

[taisau]

I have a CIFS mounted, but it has fixed itself and appears to work normally now.