Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. Long running du process keeps my backup disk spinning

Long running du process keeps my backup disk spinning

Scheduled Pinned Locked Moved Solved Support
backupsdisk-usage
11 Posts 3 Posters 1.4k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • necrevistonnezrN Offline
    necrevistonnezrN Offline
    necrevistonnezr
    wrote on last edited by joseph
    #1

    I noticed that my backup disk (external USB, my last non-SSD drive) keeps spinning all the time.
    Running fatrace -c -t on the drive, I saw a du process running this morning, probably for determining size in the System Info tab?

    05:35:43.891373 du(902292): O   /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/aclu
    05:35:43.904784 du(902292): R   /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/aclu
    05:35:43.905247 du(902292): C   /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/aclu
    05:35:43.905385 du(902292): O   /media/WD4TB/CloudronBackup/2024-09-23-220001-268/app_###_v2.17.2/data/plugins.local/af_readability/vendor/fivefilters/readability.php/test/test-pages/herald-sun-1
    

    Can I exclude this drive somehow? Could we just use df as it’s not accessing each folder and file?

    1 Reply Last reply
    2
    • nebulonN Offline
      nebulonN Offline
      nebulon
      Staff
      wrote on last edited by
      #2

      Currently there is no way to exclude drives from the stat gathering. We do run du to calculate the size each app and volume takes, this is not possible with df, however this only happens once a day https://git.cloudron.io/cloudron/box/-/blob/master/src/cron.js?ref_type=heads#L115 so not sure how long a du for all things on that disk takes. Or if other apps with data on it read/write to it on their own schedules.

      1 Reply Last reply
      1
      • necrevistonnezrN Offline
        necrevistonnezrN Offline
        necrevistonnezr
        wrote on last edited by
        #3

        Thanks for the info. However, as this drive contains an rsynced backup, there are millions of small files causing it to run forever…

        Maybe block du something like check for a NO_INDEX file in the root of the drive or something like that?
        Or use duc or ncdu as faster alternatives?

        1 Reply Last reply
        1
        • girishG Offline
          girishG Offline
          girish
          Staff
          wrote on last edited by
          #4

          Maybe we can skip du for non-ssd devices with a high inode count ? @necrevistonnezr can you get the output of df -i /backup-path ? Also, I think lsblk -d -o name,rota should display 1 for your device (indicating hdd)

          1 Reply Last reply
          1
          • necrevistonnezrN Offline
            necrevistonnezrN Offline
            necrevistonnezr
            wrote on last edited by
            #5

            Here we go:

            df -i /media/WD4TB/CloudronBackup/
            Filesystem     Inodes IUsed IFree IUse% Mounted on
            /dev/sdb1        233M  6,0M  227M    3% /media/WD4TB
            

            and

            lsblk -d -o name,rota
            NAME    ROTA
            loop0      0
            loop1      0
            loop2      0
            loop3      0
            loop4      0
            loop5      0
            loop6      0
            loop7      0
            loop8      0
            loop9      0
            loop10     0
            loop11     0
            sda        0
            sdb        1
            nvme0n1    0
            
            1 Reply Last reply
            0
            • girishG Offline
              girishG Offline
              girish
              Staff
              wrote on last edited by girish
              #6

              @necrevistonnezr nice, thank you.

              I will put some code to skip du if more than 150M inodes (arbitrary) and HDD. actually, in your case only 6M inodes are used! That's a slow disk. Maybe we should base this on read speed instead.

              How about hdparm -t /media/WD4TB ?

              1 Reply Last reply
              0
              • necrevistonnezrN Offline
                necrevistonnezrN Offline
                necrevistonnezr
                wrote on last edited by
                #7

                Cool, thank you!

                girishG 1 Reply Last reply
                0
                • necrevistonnezrN necrevistonnezr

                  Cool, thank you!

                  girishG Offline
                  girishG Offline
                  girish
                  Staff
                  wrote on last edited by
                  #8

                  @necrevistonnezr sorry, updated the post. I misread your output

                  1 Reply Last reply
                  0
                  • necrevistonnezrN Offline
                    necrevistonnezrN Offline
                    necrevistonnezr
                    wrote on last edited by
                    #9
                    hdparm -t /media/WD4TB/
                    
                    /media/WD4TB/:
                     Timing buffered disk reads: read() failed: Is a directory
                    BLKFLSBUF failed: Inappropriate ioctl for device
                    
                    1 Reply Last reply
                    0
                    • girishG Offline
                      girishG Offline
                      girish
                      Staff
                      wrote on last edited by
                      #10

                      @nebulon I guess it needs the device name /dev/xx

                      1 Reply Last reply
                      0
                      • necrevistonnezrN Offline
                        necrevistonnezrN Offline
                        necrevistonnezr
                        wrote on last edited by
                        #11

                        Yeah, I guessed the same after reading the debianforum wiki 🙂

                        Results after running it a couple of times:

                        sudo hdparm -t /dev/sdb1
                        
                        /dev/sdb1:
                         Timing buffered disk reads: 192 MB in  3.00 seconds =  63.92 MB/sec
                         Timing buffered disk reads: 200 MB in  3.00 seconds =  66.58 MB/sec
                         Timing buffered disk reads: 224 MB in  3.01 seconds =  74.44 MB/sec
                         Timing buffered disk reads: 236 MB in  3.01 seconds =  78.31 MB/sec
                         Timing buffered disk reads: 248 MB in  3.01 seconds =  82.33 MB/sec
                         Timing buffered disk reads: 234 MB in  3.01 seconds =  77.69 MB/sec 
                        
                        1 Reply Last reply
                        1
                        • girishG girish referenced this topic on
                        • girishG girish has marked this topic as solved on
                        Reply
                        • Reply as topic
                        Log in to reply
                        • Oldest to Newest
                        • Newest to Oldest
                        • Most Votes


                        • Login

                        • Don't have an account? Register

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Bookmarks
                        • Search