tarExtract pipeline error: Invalid tar header

[girish]

@nottheend said in tarExtract pipeline error: Invalid tar header:

Is there any way to do backup validation? It is probably concerning to see successful backups while they are corrupt.

This is in the immediate roadmap (https://forum.cloudron.io/topic/9975/what-s-coming-in-8-1 ). The "backup integrity". Ideally, we have to store information in the storage and also provide a mechanism to "validate" the integrity from the UI.

[jdaviescoates]

@nottheend said in tarExtract pipeline error: Invalid tar header:

It gives me the impression that the CIFS with Hetzner Storagebox is a risky choice.

I've been using it pretty much without issue for a good couple of years now, including doing a couple of complete restores.

But: I don't use encryption. I've always taken the veiw that issues relating to having troubls decrypting - eg forgetting passphrase, some corruption etc etx - are a bigger risk for me losing data than someone somehow gaining access to my Storage Boxes, in part because I don't have particularly sensitive data stored.

[potemkin_ai]

Hetzner's box, SSHFS with encryption - recovery fails with:

2024-10-19T13:37:34.456Z box:provision setProgress: restore - Downloading backup /mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc
2024-10-19T13:37:34.456Z box:storage/filesystem download: /mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc
2024-10-19T13:37:34.547Z box:backupformat/tgz Attempt 2 failed. Will retry: tarExtract pipeline error: incorrect header check

Checking the file:

file /mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc
/mnt/cloudronbackup/my_domain_cloudron/2024-10-19-130519-879/box_v8.0.6.tar.gz.enc: data

Guess encrypted data can't have tar headers?

[potemkin_ai]

in my specific case - that was due to decryption key error

[p44]

Hello everyone,

restoring Wordpress Developer instance, I had this problem right now:

May 09 13:54:41 box:backupformat/tgz tarExtract: ./data/public/wp-content/uploads/2015/02/filename.jpg 73444 file to /home/yellowtent/appsdata/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/data/public/wp-content/uploads/2015/02/filename.jpg
May 09 13:54:41 box:apptask run: app error for state pending_restore: BoxError: tarExtract pipeline error: invalid stored block lengths at tarExtract (/home/yellowtent/box/src/backupformat/tgz.js:225:26) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async /home/yellowtent/box/src/backupformat/tgz.js:248:9 at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20) at async Object.download (/home/yellowtent/box/src/backupformat/tgz.js:244:5) at async download (/home/yellowtent/box/src/backuptask.js:104:5) at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:138:5) at async install (/home/yellowtent/box/src/apptask.js:368:9) { reason: 'External Error', details: {} }
May 09 13:54:41 box:tasks setCompleted - 681: {"result":null,"error":{"stack":"BoxError: tarExtract pipeline error: invalid stored block lengths\n at tarExtract (/home/yellowtent/box/src/backupformat/tgz.js:225:26)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /home/yellowtent/box/src/backupformat/tgz.js:248:9\n at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n at async Object.download (/home/yellowtent/box/src/backupformat/tgz.js:244:5)\n at async download (/home/yellowtent/box/src/backuptask.js:104:5)\n at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:138:5)\n at async install (/home/yellowtent/box/src/apptask.js:368:9)","name":"BoxError","reason":"External Error","details":{},"message":"tarExtract pipeline error: invalid stored block lengths"}}
May 09 13:54:41 box:tasks update 681: {"percent":100,"result":null,"error":{"stack":"BoxError: tarExtract pipeline error: invalid stored block lengths\n at tarExtract (/home/yellowtent/box/src/backupformat/tgz.js:225:26)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /home/yellowtent/box/src/backupformat/tgz.js:248:9\n at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n at async Object.download (/home/yellowtent/box/src/backupformat/tgz.js:244:5)\n at async download (/home/yellowtent/box/src/backuptask.js:104:5)\n at async Object.downloadApp (/home/yellowtent/box/src/backuptask.js:138:5)\n at async install (/home/yellowtent/box/src/apptask.js:368:9)","name":"BoxError","reason":"External Error","details":{},"message":"tarExtract pipeline error: invalid stored block lengths"}}
May 09 13:54:41 box:taskworker Task took 335.441 seconds
May 09 13:54:41 BoxError: tarExtract pipeline error: invalid stored block lengths

Many others file was extracted... but then stopped with this error:

BoxError: tarExtract pipeline error: invalid stored block lengths at tarExtract

and

External Error: tarExtract pipeline error: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?

Edit:

I confirm that is something Hetzner Box related (Via CIFS).
I changed to Backblaze and everything is fine.

[nottheend]

Hopefully it will not appear with any storage anymore when this is implemented:

(9.0) Backup integrity - store size and checksum of backups. Also provide a way to "verify" backup integrity in the remote.

Mentioned in
what's coming in cloudron 9

[p44]

@nottheend ok thanks a lot

[p44]

Again, handling a "tarExtract pipeline error: incorrect header check" issue... on a different backup... very very sad for this... I hope you can release soon a solution because is risky to store backups that then cannot be restored @girish

[girish]

@p44 for encrypted backups, not sure what is the expected behaviour. If the backup is corrupt or password is forgotten, one cannot recover. What do you have in mind? What do you think Cloudron could have done? (or was your note about adding periodic integrity checks?)

[p44]

@girish

Periodic integrity checks – for example, verifying the backup archive and reporting issues proactively – could help discover corruption early, before a restore is needed. This would give admins a chance to re-run or fix backups in time.

What do you think about such a feature?