Long running du process keeps my backup disk spinning
-
wrote on Sep 27, 2024, 4:46 PM last edited by joseph Sep 30, 2024, 10:41 AM
I noticed that my backup disk (external USB, my last non-SSD drive) keeps spinning all the time.
Runningfatrace -c -t
on the drive, I saw adu
process running this morning, probably for determining size in the System Info tab?05:35:43.891373 du(902292): O /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/aclu 05:35:43.904784 du(902292): R /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/aclu 05:35:43.905247 du(902292): C /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/aclu 05:35:43.905385 du(902292): O /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/herald-sun-1
Can I exclude this drive somehow? Could we just use
df
as it’s not accessing each folder and file? -
Currently there is no way to exclude drives from the stat gathering. We do run
du
to calculate the size each app and volume takes, this is not possible withdf
, however this only happens once a day https://git.cloudron.io/cloudron/box/-/blob/master/src/cron.js?ref_type=heads#L115 so not sure how long adu
for all things on that disk takes. Or if other apps with data on it read/write to it on their own schedules. -
wrote on Sep 28, 2024, 4:39 PM last edited by
Thanks for the info. However, as this drive contains an rsynced backup, there are millions of small files causing it to run forever…
Maybe block du something like check for a
NO_INDEX
file in the root of the drive or something like that?
Or useduc
orncdu
as faster alternatives? -
Maybe we can skip du for non-ssd devices with a high inode count ? @necrevistonnezr can you get the output of
df -i /backup-path
? Also, I thinklsblk -d -o name,rota
should display 1 for your device (indicating hdd) -
wrote on Sep 30, 2024, 11:44 AM last edited by
Here we go:
df -i /media/WD4TB/CloudronBackup/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sdb1 233M 6,0M 227M 3% /media/WD4TB
and
lsblk -d -o name,rota NAME ROTA loop0 0 loop1 0 loop2 0 loop3 0 loop4 0 loop5 0 loop6 0 loop7 0 loop8 0 loop9 0 loop10 0 loop11 0 sda 0 sdb 1 nvme0n1 0
-
@necrevistonnezr nice, thank you.
I will put some code to skip du if more than 150M inodes (arbitrary) and HDD.actually, in your case only 6M inodes are used! That's a slow disk. Maybe we should base this on read speed instead.How about hdparm -t /media/WD4TB ?
-
wrote on Sep 30, 2024, 12:10 PM last edited by
Cool, thank you!
-
Cool, thank you!
@necrevistonnezr sorry, updated the post. I misread your output
-
wrote on Sep 30, 2024, 12:13 PM last edited by
hdparm -t /media/WD4TB/ /media/WD4TB/: Timing buffered disk reads: read() failed: Is a directory BLKFLSBUF failed: Inappropriate ioctl for device
-
wrote on Sep 30, 2024, 1:00 PM last edited by
Yeah, I guessed the same after reading the debianforum wiki
Results after running it a couple of times:
sudo hdparm -t /dev/sdb1 /dev/sdb1: Timing buffered disk reads: 192 MB in 3.00 seconds = 63.92 MB/sec Timing buffered disk reads: 200 MB in 3.00 seconds = 66.58 MB/sec Timing buffered disk reads: 224 MB in 3.01 seconds = 74.44 MB/sec Timing buffered disk reads: 236 MB in 3.01 seconds = 78.31 MB/sec Timing buffered disk reads: 248 MB in 3.01 seconds = 82.33 MB/sec Timing buffered disk reads: 234 MB in 3.01 seconds = 77.69 MB/sec
-
-