Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Offical apps | Community apps | Demo | Docs | Install
dsp76D

Dirk Spannaus

@dsp76
Unfollow Follow
About
Posts
221
Topics
56
Shares
0
Groups
0
Followers
1
Following
1

Posts

Recent Best Controversial

  • VPN protection for certain routes
    dsp76D dsp76
    VPN

    @luckow what a pitty. Is there another way? Would be some firewall stuff then?


  • VPN protection for certain routes
    dsp76D dsp76
    VPN

    Hi,
    are there any plans to let users adjust the VPN protection feature? It would be cool to use it for public facing apps too, but only protect the backends via VPN. Often they have a certain route - so it could be filtered by rules?

    Examples:

    • Mautic has all backend behind /s
    • WordPress has all backend behind /wp-admin
    • InvoiceNinja has public routes behind /client/

    What do you think - is this even feasible?


  • Security issue thread - PHP and eMudhra cert compromise
    dsp76D dsp76
    Security

    One thing that may be a little improvement for the future: a state where I can update a package, which is not active. So before finally activating a package - it can be updated. What do you think about this?


  • Security issue thread - PHP and eMudhra cert compromise
    dsp76D dsp76
    Security

    @Grienauer Packages are being updated one by one. Many are already patched.


  • Critical PHP RCEs (CVE-2026-109618 / CVE-2026-109619) – impact on Cloudron apps?
    dsp76D dsp76
    Security packaging

    We set our hosters firewall webhosting Ports (80/443) to selected (our own) IPs and do not allow any other traffic.
    We used a script to find out about all our apps in the cloudron containers and their PHP version. So before we switch them on again, we also need to check the app images have been updated.


  • Exclude certain paths from backup
    dsp76D dsp76
    Mastodon

    I did, its only 6 days, but the cache already hits 40GB unnecessary backup space.


  • Exclude certain paths from backup
    dsp76D dsp76
    Mastodon

    Understood - so would it mean, that maintaining the mastodon app image would need to change to store cache elsewhere, not in /app/data?


  • Exclude certain paths from backup
    dsp76D dsp76
    Mastodon

    As my mastodon app collected a lot of cached files, my daily backup got really big. I would love to have an option to exclude certain paths from backup. I could imagine other apps also create cache directories, which are not important to be in the backup?


  • Large app backup (~70GB) fails at rotate-copy step with `NoSuchKey` — IONOS S3 read-after-write consistency, but Cloudron has no retry
    dsp76D dsp76
    Support backups ionos

    Hi @james - I also received a response today. Hope its the same group of people you talk to 😉

    I let the AI compare the responses:

    Key differences between response I received and from your summary:

    • Root cause: GC overload vs. synchronous unlinking of old object data.
    • 504: Both say the proxy times out after 600s while backend processing continues.
    • HEAD vs. GET: Response I received assumes processing may still be ongoing and no data loss; from your summary says the affected data is unrecoverable.
    • CopyObject / MPU: Covered as under investigation in response I received; not addressed in your summary.
    • Fix: General GC optimization vs. explicitly decoupling GC from CompleteMultipartUpload.

    Anyway - looks like we helped uncovering the root cause 👍


  • Large app backup (~70GB) fails at rotate-copy step with `NoSuchKey` — IONOS S3 read-after-write consistency, but Cloudron has no retry
    dsp76D dsp76
    Support backups ionos

    @james thanks a lot for debugging this in detail. I pinged the IONOS Team and send them the link for this thread. Looking forward to this getting resolved. Do you still need my failed backup log? I have a fresh one from this morning 😉


  • Large app backup (~70GB) fails at rotate-copy step with `NoSuchKey` — IONOS S3 read-after-write consistency, but Cloudron has no retry
    dsp76D dsp76
    Support backups ionos

    @james IONOS Support analyzed the situation and comes back with this:

    Hello,

    Thank you for your patience. Our Platform Engineering team has reviewed and analyzed the log files you provided. Based on this analysis, we can provide the following feedback:

    1. Propagation delay: There is no artificial delay introduced by our platform. Objects become visible across different operations almost immediately, although not strictly in real time.
    2. Object consistency: Once a CompleteMultipartUpload operation returns the HTTP status code 200 OK, subsequent HeadObject requests should succeed immediately. Shortly thereafter, all other operations should also become available.

    Based on our log analysis, all multipart uploads were completed successfully. However, we observed subsequent UploadPartCopy requests referring to an UploadId that no longer exists, as the UploadId becomes invalid once an upload has been finalized.

    Could you please clarify where these UploadPartCopy requests originate from within your application or client workflow? It appears that after completion of the original upload, an automated post-processing script or background process on your side is being executed, such as a CopyObject operation.


    Could you please check now on cloudron side? Thanks a lot!


  • Login with TOTP vs Passkey
    dsp76D dsp76
    Support

    Microsoft Edge on MacOS (v 152.0.4191.66). Passkeys are stored in 1password.
    Cloudron 10.0.4


  • Login with TOTP vs Passkey
    dsp76D dsp76
    Support

    I saw there are repeatedly passkey issues, which could not be recreated. I also stumbled upon this recently together with a client. Seems like when you setup userid / password with TOTP and add passkey later, login procedure gets confused:

    • logged in with userid / password, jumped to TOTP entry mask,
    • passkey popup also appears, cancelled by user interaction
    • login doesn't proceed successfully
    • also when using at the moment the popup appears, login fails.

    Choosing "login with a passkey" works fine, but the "old way" to login with userid / pw and TOTP is broken once the passkey is set.

    When disabling TOTP for the user, the login trouble is gone. But now its the only way into the account.

    Shouldn't both work in parallel? Passkey AND UserID + PW + TOTP?


  • Excalidraw Serverside storing?
    dsp76D dsp76
    Community Packages

    I only found third party providers that add their own layer to excalidraw, as the open source community edition probably doesn't have it. I didn't yet investigate if that layer is open sourced too. Will let you know what I find.


  • Large app backup (~70GB) fails at rotate-copy step with `NoSuchKey` — IONOS S3 read-after-write consistency, but Cloudron has no retry
    dsp76D dsp76
    Support backups ionos

    Hi @james I think it means that - once Cloudron receives an error like NoSuchKey oder 404 not found when polling the object directly after upload - it still retries a number of times with exponentially growing delays added with some random delays ("Jitter"). So the error might only be temporal.


  • Excalidraw Serverside storing?
    dsp76D dsp76
    Community Packages

    Hi @timconsidine
    thanks a lot for providing Excalidraw as a community package. Do you have any plans to add a way to store draws on the server?

    Thanks!


  • Large app backup (~70GB) fails at rotate-copy step with `NoSuchKey` — IONOS S3 read-after-write consistency, but Cloudron has no retry
    dsp76D dsp76
    Support backups ionos

    IONOS responded this:

    "We are actively investigating the reported behavior. To support our Cloud Engineering team in conducting deeper log analysis within the storage cluster, we kindly request that you provide the following details:

    • Target bucket name & IONOS S3 endpoint used during testing
    • Request IDs (x-amz-request-id and x-amz-id-2 from the HTTP response headers) for both the CompleteMultipartUpload and the failed UploadPartCopy call
    • Timestamps and timezone of the affected runs (or new, millisecond-precision logs if the error can be reproduced again)

    While our Engineering team investigates the merging and propagation behavior for large multipart objects in the backend, we recommend testing one of the following client-side workarounds:

    1. HeadObject polling with exponential backoff: Before executing UploadPartCopy on newly merged objects larger than 50 GB, insert a short polling loop using HeadObject (with retries on NoSuchKey / 404 Not Found), combined with exponential backoff and jitter. This ensures object metadata is globally visible across all gateway nodes before dependent copy operations are performed.

    2. Insert a fixed delay: If implementing a polling mechanism in your Cloudron workflow is not straightforward, inserting a static delay of 1–2 seconds between CompleteMultipartUpload and subsequent copy actions can effectively prevent this race condition issue.

    Please let us know whether applying one of these workarounds resolves the issue in your backup runs, and feel free to send us the requested logs once they are available."

    I'm not sure if I can get the x-amz-request-id and x-amz-id-2 headers for them.
    What do you think about their suggestions?


  • 20GB is seemingly not enough space to upgrade from 9.2 to 10 :(
    dsp76D dsp76
    Discuss

    ... how about mounting an additional file space (temporarily) to your server? Then moving apps there for the time updating. Depending on your provider, they might be offering file space.


  • Large app backup (~70GB) fails at rotate-copy step with `NoSuchKey` — IONOS S3 read-after-write consistency, but Cloudron has no retry
    dsp76D dsp76
    Support backups ionos

    Cloudron version: 10.0.4
    Backup storage: IONOS S3-compatible Object Storage

    Symptom:
    The backup upload itself completes successfully, but the immediately following server-side rotate-copy (UploadPartCopy) fails with NoSuchKey, causing the whole backup task to fail with Old backup not found: snapshot/app_<id>.tar.gz.enc. Reproduced on two consecutive daily runs, always on our largest app (~70GB snapshot); other apps (4–70GB) in the same runs succeed.

    Timeline (Task 11372, millisecond precision):

    04:32:30.199  upload stats logged: 72,608,517,172 bytes transferred
    04:32:30.285  backupupload: upload completed. error: null   (CompleteMultipartUpload -> 200 OK)
    04:32:30.466  Copying (multipart) snapshot/app_c08ad55d-...tar.gz.enc   (rotate-copy starts, 181ms later)
    04:32:30.534  Copying part 1 - ... bytes=0-1073741823
    04:32:30.535  Copying part 2 - ... bytes=1073741824-2147483647
    04:32:30.535  Copying part 3 - ... bytes=2147483648-3221225471
    04:32:30.680  Aborting multipart copy   (145ms after part requests)
    04:32:30.748  storage/s3: copy error: NoSuchKey: UnknownError
    04:32:30.748  copy to .../app_registry.20zen.de_v1.13.0.tar.gz.enc errored. error: Old backup not found
    

    Analysis:
    CompleteMultipartUpload returns success at .285, but the immediately following UploadPartCopy on the exact same key returns NoSuchKey ~150ms later. This looks like a read-after-write consistency delay on the IONOS backend for very large multipart objects — AWS S3 itself has guaranteed strict read-after-write consistency (including for multipart uploads) since Dec 2020, so this is non-standard behavior on the storage backend side.

    That said, Cloudron's rotate-copy step (s3.js: copyInternal) has no resilience for this: no retry, no backoff, no existence check before issuing the copy. A single transient 404 immediately fails the whole backup task, even though a short retry would very likely succeed.

    Request: Could a short retry/backoff (or a HeadObject check before copying) be added around the rotate-copy step for large/multipart-uploaded objects, to tolerate brief backend propagation delays on non-AWS S3-compatible providers?

    (I will also open a ticket at IONOS, but this little improvement would make Cloudron Backups more stable for similar situations with other providers too.)

    Relevant stack:

    BoxError: Old backup not found: snapshot/app_c08ad55d-….tar.gz.enc
        at throwError (file:///home/yellowtent/box/src/storage/s3.js:568:49)
        at copyInternal (file:///home/yellowtent/box/src/storage/s3.js:636:16)
        at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
        at async Object.copy (file:///home/yellowtent/box/src/storage/s3.js:670:12)
        at async Object.copy (file:///home/yellowtent/box/src/backupformat/tgz.js:294:5)
    

    related:
    https://forum.cloudron.io/topic/14253/error-400-in-backup-process-with-ionos-s3-object-storage/38?_=1788503267379


  • Backup Integrity check runs into oom
    dsp76D dsp76
    Support backup integrity oom

    @joseph this seems not to be solved?

  • Login

  • Don't have an account? Register

  • Login or register to search.
  • First post
    Last post
0
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search