Results after running it a couple of times:

sudo hdparm -t /dev/sdb1

/dev/sdb1:
 Timing buffered disk reads: 192 MB in  3.00 seconds =  63.92 MB/sec
 Timing buffered disk reads: 200 MB in  3.00 seconds =  66.58 MB/sec
 Timing buffered disk reads: 224 MB in  3.01 seconds =  74.44 MB/sec
 Timing buffered disk reads: 236 MB in  3.01 seconds =  78.31 MB/sec
 Timing buffered disk reads: 248 MB in  3.01 seconds =  82.33 MB/sec
 Timing buffered disk reads: 234 MB in  3.01 seconds =  77.69 MB/sec 

I will put some code to skip du if more than 150M inodes (arbitrary) and HDD. actually, in your case only 6M inodes are used! That's a slow disk. Maybe we should base this on read speed instead.

How about hdparm -t /media/WD4TB ?

df -i /media/WD4TB/CloudronBackup/
Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/sdb1        233M  6,0M  227M    3% /media/WD4TB

and

lsblk -d -o name,rota
NAME    ROTA
loop0      0
loop1      0
loop2      0
loop3      0
loop4      0
loop5      0
loop6      0
loop7      0
loop8      0
loop9      0
loop10     0
loop11     0
sda        0
sdb        1
nvme0n1    0

Maybe block du something like check for a NO_INDEX file in the root of the drive or something like that?
Or use duc or ncdu as faster alternatives?