tarExtract pipeline error: Invalid tar header

[nebulon]

We have a guide in the docs on how to decrypt backups in case you haven't seen that https://docs.cloudron.io/guides/decrypt-backups/

[nottheend]

Thanks, I was not aware.
When I try to decrypt, it doesn't work, although I use the same password like for the other apps:

Error: Could not decrypt: Invalid password or tampered file (mac mismatch)

I am confused now, because I am not aware of using another password for encrypting a single app (nextcloud) compared to other apps used on the same Cloudron instance.

Is there any other idea?

[nebulon]

If you are sure the password is correct, which it would be if its the same where other backups could be decrypted, then this very much looks like the backup file is corrupt maybe redownload it from the original storage just to rule out a simple downloading error, also if you have multiple backup files maybe try one older one

[nottheend]

Thanks. Luckily I discovered that I didn't delete the old server yet. I spinned it up again and took another backup - the new backup seems to be corrupt again
Not sure what I am missing

[nebulon]

To rule out one step, maybe disable backup encryption and then see if it produces a valid tarball for a start

[nottheend]

Thanks, I am getting a different error:

An error occurred during the import operation: External Error: tarExtract pipeline error: invalid stored block lengths

Is there any limitation in terms of CIFS?

[nottheend]

I just found this hint in the docs:

Hetzner Storage Box

We recommend using SSHFS for Hetzner Storage Box since it is much faster and efficient storage wise compared to CIFS. When using Hetzner Storage Box with CIFS, the Remote Directory is /backup for the main account. For sub accounts, the Remote Directory is /subaccount.

Probably worth expanding to hint that there seem to be some unrealiable component or interaction with this way the storage box is used by Cloudron.

The issue in this thread is related to CIFS with Hetzner Storage box

[nebulon]

Do you get this error when restoring a newly made backup now without encryption? If so how large is the resulting tarball? I am not aware of any filesystem based problem with CIFS around such fundamental things as block sizes, but this is probably more related to the tar and not the filesystem unless for some reason the CIFS mount gets corrupt.

[nottheend]

Yes, I got the error mentioned before on a newly created backup without encryption. The tarball was around 13 GB.

I was only able to restore backups older than 1 month for the nextcloud app. For all other 12+ apps restoring did work like a charme for the most recent backup (younger than 1 month).

It gives me the impression that the CIFS with Hetzner Storagebox is a risky choice.
Therefore, a hint in the docs might be saving some lives of other folks.

I would like to offer 2 additional feedbacks beyond the CIFS issue and the docs:

1. it seems to be not possible to use the decryption command line cloudron command with a password that includes special characters. I tried to escape with " but it didn't work. Probably sth to document. Maybe there is a way to escape after all.
2. Is there any way to do backup validation? It is probably concerning to see successful backups while they are corrupt.

And thanks a lot for the support @girish and @nebulon for pointing me in the right direction! I resolved it by choosing a different cloud provider and different storage protocol. I used S3 now. Backing up and restoring worked with encryption completely fine!
Thank you!

[girish]

@nottheend said in tarExtract pipeline error: Invalid tar header:

Is there any way to do backup validation? It is probably concerning to see successful backups while they are corrupt.

This is in the immediate roadmap (https://forum.cloudron.io/topic/9975/what-s-coming-in-8-1 ). The "backup integrity". Ideally, we have to store information in the storage and also provide a mechanism to "validate" the integrity from the UI.

[jdaviescoates]

@nottheend said in tarExtract pipeline error: Invalid tar header:

It gives me the impression that the CIFS with Hetzner Storagebox is a risky choice.

I've been using it pretty much without issue for a good couple of years now, including doing a couple of complete restores.

But: I don't use encryption. I've always taken the veiw that issues relating to having troubls decrypting - eg forgetting passphrase, some corruption etc etx - are a bigger risk for me losing data than someone somehow gaining access to my Storage Boxes, in part because I don't have particularly sensitive data stored.

[potemkin_ai]

Hetzner's box, SSHFS with encryption - recovery fails with:

2024-10-19T13:37:34.456Z box:provision setProgress: restore - Downloading backup /mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc
2024-10-19T13:37:34.456Z box:storage/filesystem download: /mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc
2024-10-19T13:37:34.547Z box:backupformat/tgz Attempt 2 failed. Will retry: tarExtract pipeline error: incorrect header check

Checking the file:

file /mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc
/mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc: data

Guess encrypted data can't have tar headers?

[potemkin_ai]

in my specific case - that was due to decryption key error

[p44]

Hello everyone,

restoring Wordpress Developer instance, I had this problem right now:

May 09 13:54:41 box:backupformat/tgz tarExtract: ./data/public/wp-content/uploads/2015/02/filename.jpg 73444 file to /home/yellowtent/appsdata/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/data/public/wp-content/uploads/2015/02/filename.jpg
May 09 13:54:41 box:apptask run: app error for state pending_restore: BoxError: tarExtract pipeline error: invalid stored block lengths at tarExtract (/home/yellowtent/box/src/backupformat/tgz.js:225:26) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async /home/yellowtent/box/src/backupformat/tgz.js:248:9 at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20) at async Object.download (/home/yellowtent/box/src/backupformat/tgz.js:244:5) at async download (/home/yellowtent/box/src/backuptask.js:104:5) at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:138:5) at async install (/home/yellowtent/box/src/apptask.js:368:9) { reason: 'External Error', details: {} }
May 09 13:54:41 box:tasks setCompleted - 681: {"result":null,"error":{"stack":"BoxError: tarExtract pipeline error: invalid stored block lengths\n at tarExtract (/home/yellowtent/box/src/backupformat/tgz.js:225:26)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /home/yellowtent/box/src/backupformat/tgz.js:248:9\n at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n at async Object.download (/home/yellowtent/box/src/backupformat/tgz.js:244:5)\n at async download (/home/yellowtent/box/src/backuptask.js:104:5)\n at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:138:5)\n at async install (/home/yellowtent/box/src/apptask.js:368:9)","name":"BoxError","reason":"External Error","details":{},"message":"tarExtract pipeline error: invalid stored block lengths"}}
May 09 13:54:41 box:tasks update 681: {"percent":100,"result":null,"error":{"stack":"BoxError: tarExtract pipeline error: invalid stored block lengths\n at tarExtract (/home/yellowtent/box/src/backupformat/tgz.js:225:26)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /home/yellowtent/box/src/backupformat/tgz.js:248:9\n at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n at async Object.download (/home/yellowtent/box/src/backupformat/tgz.js:244:5)\n at async download (/home/yellowtent/box/src/backuptask.js:104:5)\n at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:138:5)\n at async install (/home/yellowtent/box/src/apptask.js:368:9)","name":"BoxError","reason":"External Error","details":{},"message":"tarExtract pipeline error: invalid stored block lengths"}}
May 09 13:54:41 box:taskworker Task took 335.441 seconds
May 09 13:54:41 BoxError: tarExtract pipeline error: invalid stored block lengths

Many others file was extracted... but then stopped with this error:

BoxError: tarExtract pipeline error: invalid stored block lengths at tarExtract

and

External Error: tarExtract pipeline error: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?

Edit:

I confirm that is something Hetzner Box related (Via CIFS).
I changed to Backblaze and everything is fine.

[nottheend]

Hopefully it will not appear with any storage anymore when this is implemented:

(9.0) Backup integrity - store size and checksum of backups. Also provide a way to "verify" backup integrity in the remote.

Mentioned in
what's coming in cloudron 9

[p44]

@nottheend ok thanks a lot