Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. Restore not possible when mailbox is included in backup

Restore not possible when mailbox is included in backup

Scheduled Pinned Locked Moved Solved Support
backuprestoremail
10 Posts 4 Posters 559 Views 5 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • StardenverS Offline
    StardenverS Offline
    Stardenver
    wrote on last edited by joseph
    #1

    Hi all

    Had this problem in the past, but thought I just did something wrong or a file got corrupted. But after it happened again, I did some testing. Whenever I try to restore my server from backups and a mailbox is included, it simply doesn't work. I tried it with several servers. My mailbox has about 1.7 GB - when including all mailboxes, its around 2.5GB.

    I spin up a new server, install a fresh Ubuntu LTS and I install cloudron. I then choose to do a restore and I upload the config file. In case there is no mailbox included or mailbox size is just a few MB, everything works like a charm. No problems at all. But whenever my full mailbox is included, restore fails:

    Groß (Xnip2024-12-25_22-49-26).jpeg

    I tried to restore from Hetzner VPS to Netcup VPS, Hetzner to Netcup RS, from Netcup to Hetzner VPS and from Netcup and Hetzner VPS to a dedicated Hetzner server (bare metal). In all cases the results are the same: Restore fails when a larger mailbox is included. As I do external backups of all mails, its easy to test with and without mails. Whenever I delete my mailbox and start a manual backup afterward, the restore of such backup works flawless. But once I import my mails and the mailbox is bigger than 1.x GB, the next restore will fail.

    Is there something like a file or size limit that causes restore to fail, once it reaches a specific size? Is there a way to exclude mails from a backup? I am fine backing up my mails with external tools. But in case a server crash occurs, I need to restore without any problems. At the moment I can't use and trust the Cloudron mailserver as it causes restores to fail.

    I did like 15 tests from and to different servers and its always the same result: mailbox with 1.7GB of mails = 100% restore fails - empty mailbox or just a few MB = restore works 100%

    Thanks for any help on this

    PS: I am using Hetzner Storage Box via CIFS Mount as backup storage

    1 Reply Last reply
    2
    • nebulonN Away
      nebulonN Away
      nebulon
      Staff
      wrote on last edited by
      #2

      Given that you hit a "permission denied" issue, maybe there is some race, where something changes permissions of that folder while the restore is busy copying files there. If you hit this, can you check the permisssion and ownership of the folder in question?

      robiR 1 Reply Last reply
      0
      • nebulonN nebulon marked this topic as a question on
      • C Offline
        C Offline
        CadeHenschen
        wrote on last edited by
        #3

        Hey there - I'm having a similar issue, failing on a specific mailbox restore. If I try again it claims the server is already registered, and leaves my server in a half-restored state.

        1 Reply Last reply
        0
        • nebulonN nebulon

          Given that you hit a "permission denied" issue, maybe there is some race, where something changes permissions of that folder while the restore is busy copying files there. If you hit this, can you check the permisssion and ownership of the folder in question?

          robiR Offline
          robiR Offline
          robi
          wrote on last edited by
          #4

          @nebulon can you see what flags are used for tar and make sure it's not running into uid/gid issues?

          tar should have 0 issues dropping files into a new system if it's not preserving source permissions/ownership.

          Conscious tech

          1 Reply Last reply
          0
          • nebulonN Away
            nebulonN Away
            nebulon
            Staff
            wrote on last edited by
            #5

            We don't have any more information here than what was provided in this thread, so can't quite help further for the moment

            robiR 1 Reply Last reply
            0
            • nebulonN nebulon

              We don't have any more information here than what was provided in this thread, so can't quite help further for the moment

              robiR Offline
              robiR Offline
              robi
              wrote on last edited by
              #6

              @nebulon i'm not sure you understood my comment. tar would be on the Cloudron side.

              Apart from that do you have a way to generate large multiuser mailboxes to test restores?

              Perhaps @CadeHenschen or @Stardenver can share the s3 credentials privately to test and capture the issue tar has.

              Conscious tech

              1 Reply Last reply
              0
              • nebulonN Away
                nebulonN Away
                nebulon
                Staff
                wrote on last edited by
                #7

                @robi ah you mean how the tarball is generated, this is in https://git.cloudron.io/platform/box/-/blob/master/src/backupformat/tgz.js?ref_type=heads

                But as you said, I don't think this is anywhere near the issue here.

                robiR 1 Reply Last reply
                0
                • nebulonN nebulon

                  @robi ah you mean how the tarball is generated, this is in https://git.cloudron.io/platform/box/-/blob/master/src/backupformat/tgz.js?ref_type=heads

                  But as you said, I don't think this is anywhere near the issue here.

                  robiR Offline
                  robiR Offline
                  robi
                  wrote on last edited by
                  #8

                  @nebulon well no, not generated, how it is extracted is where the error comes in, right?

                  Conscious tech

                  1 Reply Last reply
                  0
                  • nebulonN Away
                    nebulonN Away
                    nebulon
                    Staff
                    wrote on last edited by
                    #9

                    It is in that code file tarExtract(): https://git.cloudron.io/platform/box/-/blob/master/src/backupformat/tgz.js?ref_type=heads#L177

                    robiR 1 Reply Last reply
                    0
                    • nebulonN nebulon

                      It is in that code file tarExtract(): https://git.cloudron.io/platform/box/-/blob/master/src/backupformat/tgz.js?ref_type=heads#L177

                      robiR Offline
                      robiR Offline
                      robi
                      wrote on last edited by
                      #10

                      @nebulon here's what an AI found:

                      Tarball Permissions

                      The tarball contains files with ownership/permissions the process can’t replicate (e.g., root-owned files).

                      Example: Extraction fails when tar.extract preserves original ownership.

                      SELinux/AppArmor

                      Security policies block filesystem operations despite correct permissions.

                      Tar Library Behavior:
                      The tar library (likely tar-fs) might try to preserve file permissions from the tarball. If the tarball contains files owned by root and the app runs as yellowtent, extraction fails unless permissions are ignored.

                      Fix: Modify the code to ignore permissions:

                      javascript
                      
                      const tarStream = tar.extract({ cwd: destinationPath, ignore: () => true });
                      

                      Debugging Steps

                      Check Permissions:

                      javascript
                      console.log('Source:', fs.statSync(sourcePath).mode.toString(8));
                      console.log('Dest:', fs.statSync(destinationPath).mode.toString(8));
                      Ensure Directory Exists:
                      javascript
                      if (!fs.existsSync(destinationPath)) fs.mkdirSync(destinationPath, { recursive: true });
                      fs.accessSync(destinationPath, fs.constants.W_OK);
                      Add Error Logging:
                      javascript
                      const readStream = fs.createReadStream(sourcePath).on('error', (err) => console.error('Read:', err));
                      const unzipStream = zlib.createGunzip().on('error', (err) => console.error('Gunzip:', err));
                      const tarStream = tar.extract({ cwd: destinationPath }).on('error', (err) => console.error('Tar:', err));
                      

                      Potential Fixes

                      Ignore Tar Permissions:

                      javascript
                      const tarStream = tar.extract({ cwd: destinationPath, ignore: () => true });
                      

                      Test with Logging: Run the modified code with error handlers.

                      async function tarExtract(sourcePath, destinationPath) {
                          ensureSafePath(sourcePath);
                          ensureSafePath(destinationPath);
                      
                          console.log('Source:', sourcePath, fs.statSync(sourcePath));
                          console.log('Dest:', destinationPath, fs.existsSync(destinationPath) ? fs.statSync(destinationPath) : 'does not exist');
                      
                          const readStream = fs.createReadStream(sourcePath)
                              .on('error', (err) => console.error('Read error:', err));
                          const unzipStream = zlib.createGunzip()
                              .on('error', (err) => console.error('Gunzip error:', err));
                          const tarStream = tar.extract({ cwd: destinationPath })
                              .on('error', (err) => console.error('Tar error:', err));
                      
                          readStream.pipe(unzipStream).pipe(tarStream);
                      
                          await streamPromise(tarStream);
                      }
                      

                      Conclusion
                      The error is likely a mismatch between the Cloudron app’s user permissions and the filesystem.

                      Conscious tech

                      1 Reply Last reply
                      0
                      • J joseph has marked this topic as solved on
                      Reply
                      • Reply as topic
                      Log in to reply
                      • Oldest to Newest
                      • Newest to Oldest
                      • Most Votes


                      • Login

                      • Don't have an account? Register

                      • Login or register to search.
                      • First post
                        Last post
                      0
                      • Categories
                      • Recent
                      • Tags
                      • Popular
                      • Bookmarks
                      • Search