Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Slow startup caused by fixing permissions in start.sh

    Nextcloud
    2
    13
    305
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • vladimir.d
      vladimir.d last edited by vladimir.d

      Can we have an option to disable fixing file permissions on startup or fix permissions later?
      It takes 2-3 hours to start Nextcloud app (3Tb, ~2.5 million files).

      girish 1 Reply Last reply Reply Quote 3
      • Topic has been marked as a question  nebulon nebulon 
      • girish
        girish Staff @vladimir.d last edited by

        @vladimir-d Thanks, I have pushed a fix so we change permissions only when needed. We made a similar fix in minio some time ago as well.

        https://git.cloudron.io/cloudron/nextcloud-app/-/commit/33d4721e7e8fba1ac3322ef39a0a211df18cffc2

        vladimir.d 1 Reply Last reply Reply Quote 2
        • girish
          girish Staff @vladimir.d last edited by

          @vladimir-d Thanks, I have pushed a fix so we change permissions only when needed. We made a similar fix in minio some time ago as well.

          https://git.cloudron.io/cloudron/nextcloud-app/-/commit/33d4721e7e8fba1ac3322ef39a0a211df18cffc2

          vladimir.d 1 Reply Last reply Reply Quote 2
          • Topic has been marked as solved  girish girish 
          • vladimir.d
            vladimir.d last edited by

            Looks good, looking forward for a new Nextcloud release.
            Thank you.

            1 Reply Last reply Reply Quote 3
            • Referenced by  marcusquinn marcusquinn 
            • vladimir.d
              vladimir.d @girish last edited by vladimir.d

              @girish could you also review the code on lines 10-11 of start.sh.
              and can you move start.sh on writable fs?

              girish 2 Replies Last reply Reply Quote 1
              • girish
                girish Staff @vladimir.d last edited by

                @vladimir-d So far a start, I have made the chown a bit like find /app/data \( -type f -or -type d \) -and ! -user www-data -exec chown www-data:www-data '{}' \+. That essentially checks file ownership before chowing them. I have also added a flag to skip chown for restarts.

                If these two changes are not enough, then we have to add some more sophisticated check if files are already of the correct ownership or not.

                1 Reply Last reply Reply Quote 3
                • girish
                  girish Staff @vladimir.d last edited by

                  @vladimir-d Can you check if things are better with the latest update?

                  vladimir.d 2 Replies Last reply Reply Quote 1
                  • vladimir.d
                    vladimir.d @girish last edited by

                    @girish it looks promising, but a few points I need to mention about:

                    1. There is Preview Generator app which creates a cache of thumbnails in /app/data/appdata_XXXXXXXXX/preview/. It also contains lots of small files - it took 30 minutes to process 300Gb files in the folder on SSD drive (chown or even du -hs). Better to exclude it from chowning as well.

                    2. A timeout occurs on time consuming file operations (i.e. chowning) so Cloudron shows 'Not responding' app status for a quite long time (3 hours in our case). It's a little misleading. So ideally to move all such these operations to another process in the background (supervisord or even cron) and make the app running whilst it's updating permissions.

                    3. A new app release is very welcomed even if it's a dirty hotfix with start.sh on writable FS (I'd comment 'chowning' out for now).

                    1 Reply Last reply Reply Quote 1
                    • vladimir.d
                      vladimir.d @girish last edited by

                      @girish well, I've updated the app so it triggered chowning again.

                      I created the flag /run/nextcloud/chowned manually, killed the chown process and restarted the app. The upgrade downtime was just a few minutes this time.

                      Ideally to have a workaround for upgrades as well (a flag in /app/data?).

                      girish 1 Reply Last reply Reply Quote 1
                      • girish
                        girish Staff @vladimir.d last edited by girish

                        @vladimir-d awesome, thanks for testing!

                        Ideally to have a workaround for upgrades as well (a flag in /app/data?).

                        Yes, agreed, I will make a change for the release after. You should be safe for the moment across restarts/reboots.

                        The flag cannot be in /app/data because it won't work for the 'restore' case. restore is done as root user, then start.sh fixes up perms. If flag is stored in /app/data, it would incorrectly skip the chown.

                        I think what can be done is to just sample the ownership of some well known files and chown accordingly.

                        vladimir.d 1 Reply Last reply Reply Quote 0
                        • vladimir.d
                          vladimir.d @girish last edited by

                          @girish It has never ended to backup Nextcloud app using the native cloudron backup routine in our case so we disabled them for this app at all. We use rclone to backup the data to S3 for this application. Restoring would be also done using rclone running under www-data user so there will be no issues with the ownership in our case.

                          Yes, agree the workaround could be updating ownership only of certain files and folders on start.
                          Maybe a system wide background service/job (maybe using AQMP) to fix permissions/ownership files when necessary in applications?

                          1 Reply Last reply Reply Quote 0
                          • girish
                            girish Staff last edited by

                            I put a fix in https://git.cloudron.io/cloudron/nextcloud-app/-/commit/66292618931d5ecf4283f1c3fa7c8e64a68238f5

                            vladimir.d 1 Reply Last reply Reply Quote 0
                            • vladimir.d
                              vladimir.d @girish last edited by

                              @girish it looks good.
                              What would involve to move existent data to the new data directory (/app/data/data)?

                              girish 1 Reply Last reply Reply Quote 1
                              • girish
                                girish Staff @vladimir.d last edited by

                                @vladimir-d there seems to be no easy way to migrate nextcloud to different data directory (atleast in an automated way), see https://help.nextcloud.com/t/howto-change-move-data-directory-after-installation/17170 . Looks like the put the absolute path in the "share" database fields.

                                1 Reply Last reply Reply Quote 2
                                • First post
                                  Last post
                                Powered by NodeBB