Disk space usage seems incorrect on external disk

[d19dotca]

Hello,

It seems there may be an issue in 7.3 which I just noticed (unsure if this was an issue earlier), as the "Used" text doesn't seem to match in the graphic. Notice in the screenshot below that it says 200.3 GB is used, yet underneath it says Used is 189.49 GB (a difference of about 11 GB).

How is the usage calculated exactly?

I assume the lower Used number is for how much Cloudron has saved, where-as the upper Used number is perhaps an overall disk spacing query. However I think from a UX point of view, this difference in calculations can cause confusion and either needs to be explained in the UI or should just match in both numbers.

FYI - while Cloudron detects my disk size as 215.49 GB, here are the two numbers I see in terminal for it, with both base 10 and base 12 calculations:

df -h | grep -i cloudron
/dev/vdb1       216G  190G   16G  93% /mnt/cloudron-backups

df -H | grep -i cloudron
/dev/vdb1       232G  204G   17G  93% /mnt/cloudron-backups

Perhaps these numbers can be displayed in a way that matches the output from the commands above as most people likely use either of those to calculator disk usage stats.

Based on the output above, I suspect Cloudron is using the first command to calculate the size of the disk as that's closest, and the lower usage part seems right (closest again to the command output), which makes me wonder where the heck the 200.3 GB number is coming from.

[d19dotca]

Noticed the usage graph doesn't seem to change. There's absolutely an issue here of some sorts. I cleared out the disk the other day and re-started the backups yesterday to it after replacing the disk with a larger sized one. It's only using about 48 GB according to the top calculation in the image but the underneath number is much larger and the graph seems to follow that larger number which is way off.

@staff is it possible to get a response to this please? Hadn't seen one earlier, think this may have been missed.

The current df -H output shows the following (used 61 GB, free 180 GB which mostly adds up as I have a 240 GB disk):

df -H | grep cloudron
/dev/vdb1       253G   61G  180G  26% /mnt/cloudronbackup

[nebulon]

The next patch release for 7.3 will include a whole new disk usage collection, which is not based on collectd anymore and allows to refresh with live data.

[d19dotca]

@nebulon Super happy to hear that! Any ETA on when that will be released?

[girish]

@d19dotca It looks something like this now:

Among other fixes:

• It tells you when the data was collected
• Refresh to calcuate the current disk usage immediately
• We use SI units (1000) for reporting disk usage. We use IEC units (1024) for memory usage. This seems to be some industry pseudo standard.

[d19dotca]

@girish said in Disk space usage seems incorrect on external disk:

We use SI units (1000) for reporting disk usage. We use IEC units (1024) for memory usage. This seems to be some industry pseudo standard.

Perfect, thanks Girish! The si units seem to be more accurate from what I can see. I just checked on my MacBook Pro and it seems the GUI shows 494 GB of disk space and that's what the disk size is according to df -H as well, so that matches / is easier to compare.

[d19dotca]

@girish Out of curiosity... how often will it check for updated disk sizes by default without manual intervention?

Btw, I think it'd be nice if the GUI showed free space too in an easy manner instead of having us do the math ourselves haha. For your consideration...

Or maybe it could be closer to how Apple displays it where the used bit is how you have it but then in the graph itself the free bit is the available space left?

[girish]

@d19dotca said in Disk space usage seems incorrect on external disk:

@girish Out of curiosity... how often will it check for updated disk sizes by default without manual intervention?

Currently, it's manual. You have to click the refresh button on the top right to trigger a computation. I guess we can make this periodic for next release, but the issue was this value is always getting "out of date" and thus we weren't sure if there is value in periodic computation (besides, this causes much disk spin).

[d19dotca]

@girish Maybe it could be generated on page load automatically? Just a thought. But good either way.

Any possibility to get the free space calculated in the graph like it does on macOS for example? I think that'd be awesome.

[d19dotca]

@girish - I just updated and there still seems like something wrong with the calculation...169 GB used at the top of the external disk but "this disk contains" only shows 156 GB used? How is there such a big difference there when all that's on the disk is backup files from Cloudron?

Does the external disk get recalculated too when I refresh it from the top?

[d19dotca]

Update:

For my main disk...

df -H shows this for vda1 / (root):

/dev/vda1       182G  100G   73G  58% /

Yet Cloudron shows the following: (108 GB used)

Where is it getting that extra nearly 9 GB used from? The total disk size is close enough (181.37 GB vs 182 GB shown from command), it's just the usage part that seems quite inaccurate, especially if we're using si units this time with version 7.3.2.

---

For my backup disk it's similar situation:

df -H shows this for vdb1 (cloudronbackup):

/dev/vdb1       253G  205G   36G  86% /mnt/cloudronbackup

Yet Cloudron shows this: (217 GB used above graph, 204 GB used below)

In this case for the external backup disk, the lower Used number of 204.51 GB is pretty close to the 205 GB from the command, so that seems accurate, however in that case where is that extra ~12 GB coming from for the above-graph used number?

There seems to be a lot of discrepancies in the calculations here still, unfortunately. Cc @girish

[jdaviescoates]

@d19dotca said in Disk space usage seems incorrect on external disk:

Btw, I think it'd be nice if the GUI showed free space too in an easy manner instead of having us do the math ourselves

+1

Been saying this for ages

[d19dotca]

@girish - I noticed you marked this as Solved 4 days ago, but I don't believe this has actually been solved.

There's a pretty large discrepancy in usage for my external disk as seen in earlier screenshots, and I don't see that having been resolved yet.

For example, "used" under the "this disk contains:" section is 204 GB, but then the graph part shows 217 GB used. Seems like a defect that there's multiple different "used" numbers presented, and by quite a few GBs too (approximately 5 - 20 GB discrepancy).

Even in your own screenshot from earlier, the total underneath the graph adds up to about 36 GB but yet above the graph it shows 41 GB used (this is the minimum difference I've seen so far but still a large 5 GB difference in used space).

[jdaviescoates]

@d19dotca said in Disk space usage seems incorrect on external disk:

@girish - I noticed you marked this as Solved 4 days ago, but I don't believe this has actually been solved.

Also:

it'd be nice if the GUI showed free space too in an easy manner instead of having us do the math ourselves

[girish]

@d19dotca Can you give us the output of df -B1 --output=source,fstype,size,used,avail,pcent,target ? This is the command used to get disk information.

The du information is got from the du command. Can you check if running du -DsB1 <path> matches with what you see in graph for the apps and backup directory ?

Finally, for docker itself, the size comes from docker system df output.

[jdaviescoates]

@girish are you going to add the space available?!?!

[girish]

@jdaviescoates yes, your message has been relayed to @nebulon

[jdaviescoates]

@girish @nebulon thanks!

[nebulon]

The remaining available disk space is now displayed in the remainder section of the bar.

[robi]

@nebulon what happens if it fills up and shrinks the available space to print the numbers?