Memory and disk totals, how are they calculated?

[d19dotca]

I am curious how the memory and disk totals are calculated. The reason I ask is because in Cloudron it shows the following:

Memory: RAM (7.13 GB) + Swap (4.29 GB) in MB
Disk: 11.74 GB of 51.85 GB available

However, if I SSH to the server and run the free -h command for memory and the df -h command for disk space, I get different totals.

Memory:

total        used        free      shared  buff/cache   available
Mem:           6802        3887         452         535        2462        2090
Swap:          4095        1209        2886

Disk:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        49G   38G   11G  78% /

The most pressing example is my external disk for backups... it is a 64 GB block storage disk and Cloudron is calculating it at 67.51 GB, so well over 3 GB larger than it actually is.

I'm worried that the values are not being calculated correctly. Knowing how they're currently calculated would be helpful in verifying. It isn't a giant difference, but enough of a difference (particularly on the disk space front) that I'd just like to do a little digging.

[nebulon]

Regarding the memory usage, it is calculated from nodejs' os.totalmem() call and then divided by 1000 factors according to SI units. Not sure if using base 2 calculation with 1024 is more suitable or not. free -h --si should give you the same as what Cloudron calculates (with rounding due to the -h argument)

I am not exactly sure why your total memory in free comes out as 6802 though in free -h what are your server specs here. What does cat /proc/meminfo tell you?

The same applies to disk space, where we also follow SI units. But I remember @girish having dug into this in much more detail.

[girish]

The SI units (1000) vs Binary units (1024) confusion is endless. We use SI units for disk size and Binary units for memory sizes.

Please compare the outputs with df -H (not small -h), and du --block-size=1000 and so on.

For free, you shouldn't need the --si since we do use binary units.

[d19dotca]

@girish Ah interesting, I see. So I ran df -H and it does indeed show 52 GB instead of 49 GB for the root drive. This is the first I had seen that sort of command. I've always used the lowercase h for example.

Here's the thing though and where I could see there being confusion (thus the "confusion is endless" part)... you mention that you use SI units (1000) instead of binary units (1024), however that seems to only apply to the OS level queries. I say that because setting up resources on individual apps in Cloudron is all done in binary (1024) not SI units. The minimum memory for WordPress for example isn't 200, it's 256.

I think that inconsistency is possibly confusing for people, particularly since your technical audience typically deals in 1024 rather than 1000.

Is it a possibility to use the binary values in calculating disk size instead? Is there a benefit to using the SI units instead of binary?

[d19dotca]

@nebulon It's a 7 GB machine, which is probably why it's an odd number. Here's the output of the command you requested in case this helps:

ubuntu@my:~$ cat /proc/meminfo
MemTotal:        6965748 kB
MemFree:          323536 kB
MemAvailable:    2480972 kB
Buffers:          733972 kB
Cached:          1456664 kB
SwapCached:       296884 kB
Active:          2840428 kB
Inactive:        2504388 kB
Active(anon):    1754880 kB
Inactive(anon):  1901776 kB
Active(file):    1085548 kB
Inactive(file):   602612 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4194300 kB
SwapFree:        2349720 kB
Dirty:                68 kB
Writeback:             0 kB
AnonPages:       2960480 kB
Mapped:           620172 kB
Shmem:            513068 kB
Slab:            1039824 kB
SReclaimable:     761684 kB
SUnreclaim:       278140 kB
KernelStack:       33488 kB
PageTables:       137352 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     7677172 kB
Committed_AS:   15388884 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     1527664 kB
DirectMap2M:     5640192 kB

[girish]

Using SI units for disk size was not a conscious choice. I think what happened is that I used binary units when doing memory related code and @nebulon used SI units for disk related code. I think it makes sense to use binary units everywhere especially since most people are probably not going to pass --si etc to the tools. In fact, I didn't know about them until I was trying to find the discrepancy in the graphs and the tools myself.

I opened an issue for next release - https://git.cloudron.io/cloudron/box/-/issues/689

[d19dotca]

@girish That's perfect, thank you so much!