Why does refreshing Disk Usage take so long?

[jdaviescoates]

I clicked the little refresh button here:

Ages ago, but it's still just spinning around and around.

Isn't it just passing something like df -h? Why does it take so long?

[nebulon]

df -h is indeed fast, but that operates on a disk/partition level. We use that info. But for the drilled down disk usage we have to use du as well as docker commands, those take a long time and also cause extra disk I/O. This is the reason why this is updated only once a day or when manually triggered.

If anyone has suggestions on how to get this info faster on the filesystem/inode level also, this would be great.

[robi]

@nebulon since all apps are a relatively known size as packaged the only changing variables are the stored app-data from localdisk and volume mounts.

No need to keep recalculating static assets.

Hence combining the known static data with the change data size from a du -sh for very specific app dirs would likely be faster.

[nebulon]

docker image sizes are fast to retrieve also, what takes time are the app data dirs, this depends very much on disk speed as well of course on the actual used space and amount of files. The du command is already only run for specific folders.

[jdaviescoates]

@nebulon ok, thanks. No big deal really, I was just wondering.

In my case, what I actually wanted to see was if these mounted volumes had changed

Specifically the top one which is the Storage Box I do backups onto, which was getting rather full. So I changed the retention policy and did a backup cleanup, and I was just wanted to see if it had indeed freed up a load of space (It had, yay!).

I wonder if perhaps those external drives/ volumes which don't have any app data could have their own little refresh buttons too? (presumably just refreshing those wouldn't take long at all given as there is no app data to go through?)

[robi]

duc may be an option
https://duc.zevv.nl/

[robi]

From: https://unix.stackexchange.com/questions/3019/how-can-i-calculate-the-size-of-a-directory

You can use "file-size.sh" from the awk Velour library:

ls -ARgo "$@" | awk '{q += $3} END {print q}'

[girish]

Are you using a CIFS/Storage Box ? Doing a du -Dsb takes forever on those.

[jdaviescoates]

@girish said in Why does refreshing Disk Usage take so long?:

Are you using a CIFS/Storage Box ? Doing a du -Dsb takes forever on those.

Yes, ah. So perhaps it's those that is taking forever anyway, and so adding little refresh buttons just for those probably wouldn't help too much. Ah well, nevermind. No biggie.

[jdaviescoates]

I was just wondering this again and nearly created the exact same topic!

TBH it's mostly just the same results that df -h gives that I'm normally interested in (i.e. total disk usage). In this case because my backup Volume was getting full up due to so many large Nextcloud backs from lots of updates over the past week.

I'm less interested in what the specific apps are using (although that is interesting too, just rarely what I'm actually looking for)

Could the two be separated out somehow so that it's possible to just quickly get updated total df -h figures while waiting for all the detail?

[robi]

@jdaviescoates said in Why does refreshing Disk Usage take so long?:

Could the two be separated out somehow so that it's possible to just quickly get updated total df -h figures while waiting for all the detail?

This is a good idea!

[joseph]

@jdaviescoates doesn't answer your suggestion, but you can skip that backup disk from disk usage check . See the bottom of https://docs.cloudron.io/system/#disk-usage . Just add the path into /home/yellowtent/platformdata/diskusage/exclude

[jdaviescoates]

@joseph said in Why does refreshing Disk Usage take so long?:

but you can skip that backup disk from disk usage check . See the bottom of https://docs.cloudron.io/system/#disk-usage . Just add the path into /home/yellowtent/platformdata/diskusage/exclude

I don't want to skip it! It was the one I wanted to check!