Restore not possible when mailbox is included in backup

[Stardenver]

Hi all

Had this problem in the past, but thought I just did something wrong or a file got corrupted. But after it happened again, I did some testing. Whenever I try to restore my server from backups and a mailbox is included, it simply doesn't work. I tried it with several servers. My mailbox has about 1.7 GB - when including all mailboxes, its around 2.5GB.

I spin up a new server, install a fresh Ubuntu LTS and I install cloudron. I then choose to do a restore and I upload the config file. In case there is no mailbox included or mailbox size is just a few MB, everything works like a charm. No problems at all. But whenever my full mailbox is included, restore fails:

I tried to restore from Hetzner VPS to Netcup VPS, Hetzner to Netcup RS, from Netcup to Hetzner VPS and from Netcup and Hetzner VPS to a dedicated Hetzner server (bare metal). In all cases the results are the same: Restore fails when a larger mailbox is included. As I do external backups of all mails, its easy to test with and without mails. Whenever I delete my mailbox and start a manual backup afterward, the restore of such backup works flawless. But once I import my mails and the mailbox is bigger than 1.x GB, the next restore will fail.

Is there something like a file or size limit that causes restore to fail, once it reaches a specific size? Is there a way to exclude mails from a backup? I am fine backing up my mails with external tools. But in case a server crash occurs, I need to restore without any problems. At the moment I can't use and trust the Cloudron mailserver as it causes restores to fail.

I did like 15 tests from and to different servers and its always the same result: mailbox with 1.7GB of mails = 100% restore fails - empty mailbox or just a few MB = restore works 100%

Thanks for any help on this

PS: I am using Hetzner Storage Box via CIFS Mount as backup storage

[nebulon]

Given that you hit a "permission denied" issue, maybe there is some race, where something changes permissions of that folder while the restore is busy copying files there. If you hit this, can you check the permisssion and ownership of the folder in question?

[CadeHenschen]

Hey there - I'm having a similar issue, failing on a specific mailbox restore. If I try again it claims the server is already registered, and leaves my server in a half-restored state.

[robi]

@nebulon can you see what flags are used for tar and make sure it's not running into uid/gid issues?

tar should have 0 issues dropping files into a new system if it's not preserving source permissions/ownership.

[nebulon]

We don't have any more information here than what was provided in this thread, so can't quite help further for the moment

[robi]

@nebulon i'm not sure you understood my comment. tar would be on the Cloudron side.

Apart from that do you have a way to generate large multiuser mailboxes to test restores?

Perhaps @CadeHenschen or @Stardenver can share the s3 credentials privately to test and capture the issue tar has.

[nebulon]

@robi ah you mean how the tarball is generated, this is in https://git.cloudron.io/platform/box/-/blob/master/src/backupformat/tgz.js?ref_type=heads

But as you said, I don't think this is anywhere near the issue here.

[robi]

@nebulon well no, not generated, how it is extracted is where the error comes in, right?

[nebulon]

It is in that code file tarExtract(): https://git.cloudron.io/platform/box/-/blob/master/src/backupformat/tgz.js?ref_type=heads#L177

[robi]

@nebulon here's what an AI found:

Tarball Permissions

The tarball contains files with ownership/permissions the process can’t replicate (e.g., root-owned files).

Example: Extraction fails when tar.extract preserves original ownership.

SELinux/AppArmor

Security policies block filesystem operations despite correct permissions.

Tar Library Behavior:
The tar library (likely tar-fs) might try to preserve file permissions from the tarball. If the tarball contains files owned by root and the app runs as yellowtent, extraction fails unless permissions are ignored.

Fix: Modify the code to ignore permissions:

javascript

const tarStream = tar.extract({ cwd: destinationPath, ignore: () => true });

Debugging Steps

Check Permissions:

javascript
console.log('Source:', fs.statSync(sourcePath).mode.toString(8));
console.log('Dest:', fs.statSync(destinationPath).mode.toString(8));
Ensure Directory Exists:
javascript
if (!fs.existsSync(destinationPath)) fs.mkdirSync(destinationPath, { recursive: true });
fs.accessSync(destinationPath, fs.constants.W_OK);
Add Error Logging:
javascript
const readStream = fs.createReadStream(sourcePath).on('error', (err) => console.error('Read:', err));
const unzipStream = zlib.createGunzip().on('error', (err) => console.error('Gunzip:', err));
const tarStream = tar.extract({ cwd: destinationPath }).on('error', (err) => console.error('Tar:', err));

Potential Fixes

Ignore Tar Permissions:

javascript
const tarStream = tar.extract({ cwd: destinationPath, ignore: () => true });

Test with Logging: Run the modified code with error handlers.

async function tarExtract(sourcePath, destinationPath) {
    ensureSafePath(sourcePath);
    ensureSafePath(destinationPath);

    console.log('Source:', sourcePath, fs.statSync(sourcePath));
    console.log('Dest:', destinationPath, fs.existsSync(destinationPath) ? fs.statSync(destinationPath) : 'does not exist');

    const readStream = fs.createReadStream(sourcePath)
        .on('error', (err) => console.error('Read error:', err));
    const unzipStream = zlib.createGunzip()
        .on('error', (err) => console.error('Gunzip error:', err));
    const tarStream = tar.extract({ cwd: destinationPath })
        .on('error', (err) => console.error('Tar error:', err));

    readStream.pipe(unzipStream).pipe(tarStream);

    await streamPromise(tarStream);
}

Conclusion
The error is likely a mismatch between the Cloudron app’s user permissions and the filesystem.