@nichu42 said:

I have an 8 Core Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz VPS.
My Mastodon app data directory has ~ 450,000 files.

FYI my yellowtent folder has ~ 250.000 files on an SSD - just to give you perspective on the benchmarks.

But changing to gdu as suggested by @necrevistonnezr seems to be a good idea anyway.

du

time (du -h --dereference-args --summarize --block-size=1 /home/yellowtent/ | sort -hr) 

„Warm Cache“:
real  0m0,617s
user  0m0,139s
sys   0m0,482s

After reboot (to simulate cold cache):
real  0m12,146s
user  0m0,347s
sys   0m1,733s

gdu

time (gdu -ns -a --no-prefix -L -I /home/yellowtent/ | awk '{print $1"\t"$2}')

„Warm cache“:
real  0m0,239s
user  0m0,391s
sys   0m0,372s

After reboot (to simulate cold cache):
real  0m2,844s
user  0m0,729s
sys   0m1,652s

Multi-threaded Alternatives:

• gdu (in repositories): Go-based, multi-threaded.
• diskus (Rust, GitHub) : Minimalist CLI total-size calculator.

Disk Usage Benchmark Comparison

Dataset: ~400,000 small files across 100,000 directories (~15 GB on NVMe SSD, 8-Core CPU)

On non-cached disks (e.g., night-time cron tasks), multi-threaded tools (gdu, diskus) bypass the stat() I/O latency bottleneck by querying drive metadata across all CPU threads concurrently.

NOT a full drop-In Replacement for Scripts

Command Comparison
Original GNU du

du --dereference-args --summarize --block-size=1 --exclude=*.lock --exclude=dovecot.list.index.log.* /home/yellowtent/

Script-Compatible gdu Drop-In

gdu -ns -a --no-prefix -L -I ".*\.lock,dovecot\.list\.index\.log.*" /home/yellowtent/ | awk '{print $1"\t"$2}'

Flag Mapping

Key Caveats

• Delimiter (Tab vs. Space): GNU du separates bytes and path with \t. gdu uses spaces. Piping to awk '{print $1"\t"$2}' prevents broken cut -f1 parsers in scripts.
• Symlink Scope: GNU du --dereference-args resolves symlinks only at the root level. gdu -L resolves all internal symlinks recursively, which can inflate total calculated sizes.
• Exit Codes: gdu -n silently skips unreadable files and exits 0. GNU du exits 1 on permission errors, which backup scripts often monitor.