SSH remote copy always failed, falling back to sshfs copy

[Dummyzam]

Thanks for pointing this because no, i didn't.
But now yes, and i have the same issue.
/tmp/identity_file... was not found but i was able to log with password (i need to remove that).
I did the test with an absolute path and it woks.
I'm not sure how i should interpret all of it... Is this an issue similar to this topic ? Have i set-up sshfs target incorrectly causing the relative path to fail ? Is this another connection/permission issue as i saw on another post i can't find ?

[jadudm]

Is Cloudron using a correct/full path when issuing the cp over ssh? I've had this problem, too, and I know I'm not spanning ZFS pools. If we enter a root path into the config, should't Cloudron use the full paths, for correctness/completeness/clarity, when issuing the cp?

I don't think that helps much, but I've seen the same persistent issues that @dummyzam is pointing to.

[joseph]

yeah, it's always absolute paths.

[Dummyzam]

thanks @joseph for your answer.
But the command you gave me (same one as in error log) use relative path.
And when i try the same command manually with an absolute path, it seems to works.
What should i understand of it ?

[joseph]

@Dummyzam you are right. For sshfs, the paths are relative (to the user's home). The cp command is run after ssh into that server.

I did the test with an absolute path and it woks.

What did you mean by this? I think we have to figure out why ssh -p 22 myuser@192.168.0.10 cp -aRl snapshot/app_6da85c9a-fbf0-4563-baaa-1ad3080cf467 temp does not work. Is that because the ssh user has some other home directory? Maybe your prefix (in the backup site config) is an absolute path and not a relative path?

[Dummyzam]

@joseph said:

he paths are relative (to the user's home)

Is it relative to user $home or to target directory ?
Because if it is to the target directory, it should have been '../CloudronBackup/snapshot' in my case and not 'snapshot'

Because

@joseph said:
What did you mean by this?

My user $home is : /mnt/OnlyZpol/UserTst/
My target directory is : /mnt/OnlyZpol/CloudronBackup/

Running :

ssh -o "StrictHostKeyChecking no" -p 22 myuser@192.168.0.10 cp -aRl snapshot/app_6da85c9a-fbf0-4563-baaa-1ad3080cf467 temp

-> FAIL

ssh -o "StrictHostKeyChecking no" -p 22 myuser@192.168.0.10 cp -aRl /mnt/OnlyZpol/CloudronBackup/snapshot/app_6da85c9a-fbf0-4563-baaa-1ad3080cf467 /mnt/OnlyZpol/CloudronBackup/temp

-> Success

Is that because the ssh user has some other home directory? Maybe your prefix (in the backup site config) is an absolute path and not a relative path?

So yes for the first and no for the second, i never set an "absolute prefix".
I will try to use a subdirectory of user's home, but i've already tried quite similar with a sub-dataset ; without luck.

IMO it whould be safer to give absolute path in sshfs as you can't really ensure ssh will login each time in same directory

[joseph]

@Dummyzam I think you can set the prefix to /mnt/OnlyZpol/CloudronBackup . That will make cp use absolute paths.

[Dummyzam]

@joseph thanks for the interresting idea but Cloudron does not allow it.
On back up setup it shows this message : "prefix must be a relative path"

[jadudm]

I'm not sure where in the Box codebase this is, but the SSH backup behavior is strange. (Or, not documented sufficiently clearly for me to make sense of it.) I spent a bunch of time trying to figure this out as well, and ultimately gave up. However, @dummyzam is encountering many of the same kinds of confusion I did.

Ideally:

1. The remote directory should be the base for all operations, as far as backup site configuration is concerned.
2. All operations should be absolute paths, and be rooted at join(remote dir, prefix). No backup operations should take place outside of that root path on the remote system.

If those things are true, then it should be the case that given a target/remote directory of (say) /mnt/OnlyZpol/CloudronBackup/ and a prefix of backups, then all operations will be against /mnt/OnlyZpol/CloudronBackup/backups/*.

If $HOME is used by Box when doing SSHFS backups, then that should be documented somewhere.

As I learned, the target/remote directory will be set to 777, which can be a problem if the user you're authenticating as lacks permissions, or if you make the mistake of using $HOME as your remote directory (as this can upset the permissions that SSHD expects for .ssh).

[nebulon]

So for sshfs mostly we just do a local sshfs (fuse) mount point and then the backup strategy runs exactly the same way as for all other backup targets. So that code is not specific. However if you use rsync with hardlink support, then sshfs really can shine and for that we have added specific code paths to do the remote ssh login and create the hardlinks on the server directly, which is the most common operation for incremental backups. This speeds up the backup process a lot.

I am writing this, since above it seems to be about that optimized code path, which is doing the ssh login first and then runs the cp on the remote host (not your Cloudron).