Restore not possible

[jdaviescoates]

I can't help, but just to say this stuff is part of why I tend not to use encryption as in my limited experience the risk of my locking myself out of my own data is higher than the risk that someone else will access it!

[james]

Hello @sponch
Can you please try to manually decrypt a singular backup following this guide:
https://docs.cloudron.io/guides/decrypt-backups/

[sponch]

TypeError: Invalid initialization vector
    at Decipheriv.createCipherBase (node:internal/crypto/cipher:117:19)
    at Decipheriv.createCipherWithIV (node:internal/crypto/cipher:128:3)
    at new Decipheriv (node:internal/crypto/cipher:247:3)
    at Object.createDecipheriv (node:crypto:145:10)
    at decryptFilePath (/opt/homebrew/lib/node_modules/cloudron/src/backup-tools.js:55:36)
    at Command.decryptDir (/opt/homebrew/lib/node_modules/cloudron/src/backup-tools.js:284:50)
    at Command.listener [as _actionHandler] (/opt/homebrew/lib/node_modules/cloudron/node_modules/commander/lib/command.js:552:17)
    at /opt/homebrew/lib/node_modules/cloudron/node_modules/commander/lib/command.js:1586:14
    at Command._chainOrCall (/opt/homebrew/lib/node_modules/cloudron/node_modules/commander/lib/command.js:1470:12)
    at Command._parseCommand (/opt/homebrew/lib/node_modules/cloudron/node_modules/commander/lib/command.js:1585:27) {
  code: 'ERR_CRYPTO_INVALID_IV'

[james]

Is this a tgz or rsync backup?

[sponch]

@james rsync

[james]

@sponch
Once, I've seen a strange error with rsync type backup where a zero byte file blocked the whole process.
Because, a zero byte file does not even include the encryption header needed for the file.

Can you please run the following command for the failing rsync decryption:

find . -type f -size 0

If you find any zero byte files, move them somewhere else outside the folder and try the decryption again.

[sponch]

a run the command in the encrypted backup directly right? doesn't find anything

[sponch]

what would you recommend: rsync or tgz for future backups?

[p44]

We discussed this issue here:
https://forum.cloudron.io/topic/12632/tarextract-pipeline-error-invalid-tar-header/24

I've encountered this problem twice...

@sponch I have a question: Were the backups you're trying to restore still in their original location (where they were first created), or had they been moved/copied to a different Hetzner Box before restoring?

I’m asking because the files I tried to recover (unsuccessfully) had been transferred between Hetzner Boxes, and I’m wondering if that could be the root of the issue.

Edit: Were the files you tried to restore from the Hetzner Box originally created by the same specific Cloudron instance? Or were they from a different instance (e.g., after a reinstall)? I’m asking to help trace the root cause of the issue.

@james I believe that in some cases encryption is necessary, so increased security inherently comes with the trade-off of being locked out of the recovery process. However, in this specific case, we should investigate the root cause of the issue, since it seems that has affected multiple users...

@sponch @james Another hypothesis that comes to mind is the use of special characters in the password, which might not be interpreted correctly during the backup recovery process – Eg. a Gui problem –. I’ve since updated my passwords to only include uppercase letters, lowercase letters, and numbers (40 characters long), and the issue appears to be resolved... but this is only an hypothesis.

[sponch]

@sponch I have a question: Were the backups you're trying to restore still in their original location (where they were first created), or had they been moved/copied to a different Hetzner Box before restoring?

Same location (only the prefixed changed months ago - should not concern the recent backups)

I’m asking because the files I tried to recover (unsuccessfully) had been transferred between Hetzner Boxes, and I’m wondering if that could be the root of the issue.

Edit: Were the files you tried to restore from the Hetzner Box originally created by the same specific Cloudron instance? Or were they from a different instance (e.g., after a reinstall)? I’m asking to help trace the root cause of the issue.

Same instance. Nextcloud is backed up daily. Last fews days restoring always ends up with same error.

@sponch @james Another hypothesis that comes to mind is the use of special characters in the password, which might not be interpreted correctly during the backup recovery process – Eg. a Gui problem

Just restored another (unimportant) app from same backup day. Worked perfectly.

[p44]

@sponch ok thanks for informations. I hope those informations could help to reproduce issue and eventually find the source cause...

[sponch]

hi and thanks four you information. Unfortunately I could not find the cause.... and set up a new Nextcloud instance (only had 5 users and offline data from the clients).
As other apps could be restored w/o any problems it must have been s.th. related to Nextcloud specific restoration I guess.
Just made a clone of the new Nextcloud App (encryption disabled....)worked perfectly.

[p44]

@sponch you disabled Cloudron Backup Encryption or Nextcloud App Encription?

[Kelsey67]

@sponch I learned the hard way and lost a ton of data using tgz. Rsync is definitely better, i dont even know why anyone would both with tgz. Unless you combine them, use rsync for daily incremental backups to a remote server and tgz for weekly/monthly full archives. i have a large Nextcloud data directory on an AWS server, managed HostJane EC2 - https://www.hostjane.com: supporting 500MB of changes daily (e.g., new uploads, modified files) needing frequent backups for tens of GBs for mainly photos). I like to set up rsync via Cygwin, automate it with Task Scheduler, and pair it with database dumps.

e.g. Rsync transfers the full 50GB data directory to the NAS.
Time: ~2 hours at 5MB/s (50GB ÷ 5MB/s = 10,000 seconds ≈ 167 minutes).
Bandwidth: 50GB.

Compare that to tgz. Backup takes ~6 hours 20 minutes, regardless of changes, due to full compression and transfer. tgz completely screws up Nextcloud availability if maintenance mode extends into active hours. tgz requires a full archive each time, additional steps to manage multiple archives. tgz is redundant for every use case of every Nextcloud backup I have ever done.

[sponch]

@p44 I haven't uses NC Encryption. I disabled backup encryption now (which doesn't feel right....). But as I said - other apps could be restored w/o any problems.
As restoring backups should be one of the most important thing for everyone here I wonder why there came no further support from "official" side here.

[sponch]

@Kelsey67 thanks. that’s what I have used anyway.

[p44]

@sponch Thanks, keep us posted.

[necrevistonnezr]

restic is made for this. Fast, e2e encrypted, any provider you can imagine via rclone.

[p44]

@necrevistonnezr Thanks a lot, it seems to be a very good strategy, but implementation is out of my personal knowledge.