[OT] Hetzner Box night backup takes 6 hours?!

[jdaviescoates]

@p44 I think others on the forum have reported much faster rsync backups with sshfs.

I don't currently use rsync for backups myself.

[p44]

@jdaviescoates Thanks. I have never never had any problems with current setup, but for about 20 days now I have had this night and day speed difference.

[andreasdueren]

@p44 said in [OT] Hetzner Box night backup takes 6 hours?!:

@jdaviescoates Interesting... I have to say that I have backup on two different data centers, from Nuremberg to Helsinki

I do the same and have excellent performance. Try SSH

[p44]

@andreasdueren Thanks I will test

[joseph]

As @andreasdueren suggested, use sshfs when possible with hetzner storage box. The remote copy is pretty much instantaneous because the code sshs into the server and executes a copy command.

[p44]

Thanks @joseph I'll follow these advices

[jayonrails]

@joseph said in [OT] Hetzner Box night backup takes 6 hours?!:

As @andreasdueren suggested, use sshfs when possible with hetzner storage box. The remote copy is pretty much instantaneous because the code sshs into the server and executes a copy command.

What would the exact configuration look like?
Can you share yours?

[andreasdueren]

@jayonrails Sure:

Storage Provider: SSHFS Mount
Server IP or Hostname (sshfs): uxxxxxx.your-storagebox.de
Remote Directory (sshfs): /home
Port: 23
User: uxxxxxx-sub2 # I use a sub account
Private Key: set # Make sure to upload the respective public key to the directory
Prefix: empty
Storage Format: rsync
Encryption Password (optional): set

[p44]

@andreasdueren In SSHFS Cloudron help page I read "When using Hetzner Storage Box, the Remote Directory is /home for the main account. We have found sub accounts to be unreliable with SSHFS. We recommend using CIFS instead if you want to use subaccounts."

Did you have some problems with hetzner sub accounts?

Also, when you write "Make sure to upload the respective public key to the directory", what do you mean exactly? In "Private Key" field what I have to paste? Then, what I have to put in directory you mention? Do you mean Hetzner sub account directory?

Edit: @andreasdueren I found something here about Ssh Key

Thanks for your time!

[jdaviescoates]

@p44 said in [OT] Hetzner Box night backup takes 6 hours?!:

Edit: @andreasdueren I found something here about Ssh Key

This is a key bit of info there:

As an alternative, with the parameter -f, you can specify a different file path.

So, on you local machine use that to generate a pair of keys in e.g. ./ssh-backups/ (instead of over writing what is in ./ssh/ ) then follow the instructions to upload the id_rsa.pub public key file onto the Storage Box and paste the id_rsa private key into the config box.

To be triply clear this is what I do:

On your local machine:

1. mkdir ~/.ssh-backups
2. ssh-keygen -f ~/.ssh-backups/id_rsa
3. cat ~/.ssh-backups/id_rsa.pub | ssh -p23 uXXXXX@uXXXXX.your-storagebox.de install-ssh-key

NOTE: of course that final command will only work if you've already got SSH enabled on your Hetzner Storage Box via the settings here https://robot.hetzner.com/storage

Via the Cloudron UI when setting up a backup location (or e.g. a volume as the case may be):

4. Paste the contents of ~/.ssh-backups/id_rsa into the Private Key field here

It took ages to work that out, but it's easy once you know how.

I also then store both keys in my VaultWarden in case I need them again.

[andreasdueren]

@p44 No Problems with the sub account whatsoever

[p44]

Thanks a lot @jdaviescoates @andreasdueren

[jdaviescoates]

@p44 I updated my post above with a few more details

[jdaviescoates]

But, for the record, I updated my backups config to use SSHFS instead of CIFS earlier and it seem using Tarball backups SSHFS isn't any faster at all. If anything it seems like perhaps it's a bit slower (at least the one that ran tonight was slower than 6 out of 8 of the last CIFS backups). I've read on here that rsync backups are much faster with SSHFS though.

I'm looking forward to when we can do multiple backup locations, which I'm hoping will allow me to do rsync to one destination and tarball to another.

[robi]

@jdaviescoates the rsync may be doing a 'cat' stream over ssh vs the tgz having to complete locally before transferring.

Hmm, wasn't there a way to do that with tgz as well?

[p44]

wow @jdaviescoates great!!! I really appreciate.

[p44]

@jdaviescoates said in [OT] Hetzner Box night backup takes 6 hours?!:

I'm looking forward to when we can do multiple backup locations, which I'm hoping will allow me to do rsync to one destination and tarball to another.

Yes, waiting for multiple backup locations.