Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

Cloudron Forum

Apps | Demo | Docs | Install

Slow startup caused by fixing permissions in start.sh

Scheduled Pinned Locked Moved Solved Nextcloud
13 Posts 2 Posters 371 Views
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    wrote on last edited by vladimir.d
    #1

    Can we have an option to disable fixing file permissions on startup or fix permissions later?
    It takes 2-3 hours to start Nextcloud app (3Tb, ~2.5 million files).

    girishG 1 Reply Last reply
    3
  • nebulonN nebulon marked this topic as a question on
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to vladimir.d on last edited by
    #0

    @vladimir-d Thanks, I have pushed a fix so we change permissions only when needed. We made a similar fix in minio some time ago as well.

    https://git.cloudron.io/cloudron/nextcloud-app/-/commit/33d4721e7e8fba1ac3322ef39a0a211df18cffc2

    vladimir.dV 1 Reply Last reply
    2
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to vladimir.d on last edited by
    #2

    @vladimir-d Thanks, I have pushed a fix so we change permissions only when needed. We made a similar fix in minio some time ago as well.

    https://git.cloudron.io/cloudron/nextcloud-app/-/commit/33d4721e7e8fba1ac3322ef39a0a211df18cffc2

    vladimir.dV 1 Reply Last reply
    2
  • girishG girish has marked this topic as solved on
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    wrote on last edited by
    #3

    Looks good, looking forward for a new Nextcloud release.
    Thank you.

    1 Reply Last reply
    3
  • marcusquinnM marcusquinn referenced this topic on
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    replied to girish on last edited by vladimir.d
    #4

    @girish could you also review the code on lines 10-11 of start.sh.
    and can you move start.sh on writable fs?

    girishG 2 Replies Last reply
    1
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to vladimir.d on last edited by
    #5

    @vladimir-d So far a start, I have made the chown a bit like find /app/data \( -type f -or -type d \) -and ! -user www-data -exec chown www-data:www-data '{}' \+. That essentially checks file ownership before chowing them. I have also added a flag to skip chown for restarts.

    If these two changes are not enough, then we have to add some more sophisticated check if files are already of the correct ownership or not.

    1 Reply Last reply
    3
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to vladimir.d on last edited by
    #6

    @vladimir-d Can you check if things are better with the latest update?

    vladimir.dV 2 Replies Last reply
    1
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    replied to girish on last edited by
    #7

    @girish it looks promising, but a few points I need to mention about:

    1. There is Preview Generator app which creates a cache of thumbnails in /app/data/appdata_XXXXXXXXX/preview/. It also contains lots of small files - it took 30 minutes to process 300Gb files in the folder on SSD drive (chown or even du -hs). Better to exclude it from chowning as well.

    2. A timeout occurs on time consuming file operations (i.e. chowning) so Cloudron shows 'Not responding' app status for a quite long time (3 hours in our case). It's a little misleading. So ideally to move all such these operations to another process in the background (supervisord or even cron) and make the app running whilst it's updating permissions.

    3. A new app release is very welcomed even if it's a dirty hotfix with start.sh on writable FS (I'd comment 'chowning' out for now).

    1 Reply Last reply
    1
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    replied to girish on last edited by
    #8

    @girish well, I've updated the app so it triggered chowning again.

    I created the flag /run/nextcloud/chowned manually, killed the chown process and restarted the app. The upgrade downtime was just a few minutes this time.

    Ideally to have a workaround for upgrades as well (a flag in /app/data?).

    girishG 1 Reply Last reply
    1
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to vladimir.d on last edited by girish
    #9

    @vladimir-d awesome, thanks for testing!

    Ideally to have a workaround for upgrades as well (a flag in /app/data?).

    Yes, agreed, I will make a change for the release after. You should be safe for the moment across restarts/reboots.

    The flag cannot be in /app/data because it won't work for the 'restore' case. restore is done as root user, then start.sh fixes up perms. If flag is stored in /app/data, it would incorrectly skip the chown.

    I think what can be done is to just sample the ownership of some well known files and chown accordingly.

    vladimir.dV 1 Reply Last reply
    0
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    replied to girish on last edited by
    #10

    @girish It has never ended to backup Nextcloud app using the native cloudron backup routine in our case so we disabled them for this app at all. We use rclone to backup the data to S3 for this application. Restoring would be also done using rclone running under www-data user so there will be no issues with the ownership in our case.

    Yes, agree the workaround could be updating ownership only of certain files and folders on start.
    Maybe a system wide background service/job (maybe using AQMP) to fix permissions/ownership files when necessary in applications?

    1 Reply Last reply
    0
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    wrote on last edited by
    #11

    I put a fix in https://git.cloudron.io/cloudron/nextcloud-app/-/commit/66292618931d5ecf4283f1c3fa7c8e64a68238f5

    vladimir.dV 1 Reply Last reply
    0
  • vladimir.dV Offline
    vladimir.dV Offline
    vladimir.d
    replied to girish on last edited by
    #12

    @girish it looks good.
    What would involve to move existent data to the new data directory (/app/data/data)?

    girishG 1 Reply Last reply
    1
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to vladimir.d on last edited by
    #13

    @vladimir-d there seems to be no easy way to migrate nextcloud to different data directory (atleast in an automated way), see https://help.nextcloud.com/t/howto-change-move-data-directory-after-installation/17170 . Looks like the put the absolute path in the "share" database fields.

    1 Reply Last reply
    2

  • Login

  • Don't have an account? Register

  • Login or register to search.
  • First post
    Last post
0
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Login

  • Don't have an account? Register

  • Login or register to search.