Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

Cloudron Forum

Apps | Demo | Docs | Install

Consider improvements to the backup experience to support long filenames/directory names

Scheduled Pinned Locked Moved Feature Requests
backupsencryption
11 Posts 4 Posters 446 Views
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R Offline
    R Offline
    Robin
    wrote on last edited by girish
    #1

    Large filenames cause problems, especially with blob-based storage (e.g. S3-compatible services have a key limit of 1024). This gives problems quite easily with hierarchical storage like syncthing where one might have a set of nested folders that quite easily hit that key limit, especially combined with encryption.

    Disabling encryption altogether is not desirable in many cases, but it could be worthwhile to allow disabling filename encryption, in some cases.

    (As a longer term project, it may also be worth considering ways to circumvent this limit altogether.)

    See https://forum.cloudron.io/topic/3050/backup-feedback-minio/ for the initial discussion leading to this request.

    1 Reply Last reply
    4
  • girishG Offline
    girishG Offline
    girish Staff
    wrote on last edited by
    #2

    If someone knows how other tools handle it would be great. I know rclone has similar restrictions as us - https://rclone.org/crypt/#name-encryption

    Jan MacenkaJ 1 Reply Last reply
    0
  • Jan MacenkaJ Offline
    Jan MacenkaJ Offline
    Jan Macenka
    replied to girish on last edited by
    #3

    @girish is there any progress here? Wanted to implement CR for a business client and ran into this problem.

    Since the NextCloud Instance already has approx 1 TB and increasing, doing daily (request by client) tarball Backups to an offsite storage produces way too much traffic. rsync would be perfect and the Client already has MINIO services running but they encounter the already discussed key-length-error when syncing.

    Any update here? Is there something on the roadmap to solve this?

    girishG 2 Replies Last reply
    0
  • girishG Offline
    girishG Offline
    girish Staff
    replied to Jan Macenka on last edited by
    #4

    @Jan-Macenka We can look into this for the next release but for the moment the best workaround is to rename the files. Alternately, don't use minio and instead use some sort of filesystem mount (nfs) setup.

    1 Reply Last reply
    0
  • girishG Offline
    girishG Offline
    girish Staff
    replied to Jan Macenka on last edited by
    #5

    @Jan-Macenka @Robin Would it be acceptable if filename encryption was optional? That would atleast make it slightly better.

    Jan MacenkaJ 1 Reply Last reply
    0
  • Jan MacenkaJ Offline
    Jan MacenkaJ Offline
    Jan Macenka
    replied to girish on last edited by
    #6

    @girish as a quick fix this would help aslong as the contents still are encrypted 🙂

    girishG 1 Reply Last reply
    1
  • girishG Offline
    girishG Offline
    girish Staff
    replied to Jan Macenka on last edited by
    #7

    @Jan-Macenka ok, that should be easy to do for next release. a checkbox that disables filename encryption.

    1 Reply Last reply
    2
  • marcusquinnM Offline
    marcusquinnM Offline
    marcusquinn
    wrote on last edited by
    #8

    I wonder if filenames/paths could be hashed into a database lookup table?

    I've not looked in greater detail, but I suspect this how blockchain file storage systems like IFPS and Storj might handle such things.

    We're not here for a long time - but we are here for a good time :)
    Jersey/UK
    Work & Ecommerce Advice: https://brandlight.org
    Personal & Software Tips: https://marcusquinn.com

    girishG 1 Reply Last reply
    1
  • girishG Offline
    girishG Offline
    girish Staff
    replied to marcusquinn on last edited by
    #9

    @marcusquinn yes, that's the long term plan. The implementation is a bit involved though. This "database" has to be stored as a separate file/object on s3, downloaded and then we have to decrypt filenames. And viceversa for the backup process where we have to upload this database after all files are backed up.

    marcusquinnM 1 Reply Last reply
    1
  • marcusquinnM Offline
    marcusquinnM Offline
    marcusquinn
    replied to girish on last edited by
    #10

    @girish Yeah, that's the fun part, easier said than done eh — especially when you have to consider unexpected interrupts & resumes etc.

    We're not here for a long time - but we are here for a good time :)
    Jersey/UK
    Work & Ecommerce Advice: https://brandlight.org
    Personal & Software Tips: https://marcusquinn.com

    1 Reply Last reply
    0
  • girishG Offline
    girishG Offline
    girish Staff
    wrote on last edited by
    #11

    @Jan-Macenka OK, I have made filename encryption optional in 7.3.

    fbb205df-5917-403b-92c5-9084a856c656-image.png

    1 Reply Last reply
    3

  • Login

  • Don't have an account? Register

  • Login or register to search.
  • First post
    Last post
0
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Login

  • Don't have an account? Register

  • Login or register to search.