Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Navigation

    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Backup feedback over sshfs

    Support
    backups sshfs
    2
    4
    33
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      Robin last edited by girish

      Some feedback on trying to use sshfs as a backup mechanism. Setup was: cloudron node connected to a remote server hosted in a different country using sshfs.

      • Using the rsync method, this was unusably slow, likely due to smaller file sizes (~10 minutes just to process a G or two of data, despite having a decent internet connection on both ends).
      • Using tarballs was a bit of a better experience, the backup did eventually complete, but this is not really a practical possibility given the size of data I was trying to back up at present (as I understand it, basically, each day would involve a full upload of those tarballs).

      Not sure what can really be done about this from the Cloudron side without some pretty serious surgery, but thought that it was worth mentioning.

      1 Reply Last reply Reply Quote 0
      • girish
        girish Staff last edited by

        For the last release, I did some basic performance tests to get the backup speed to be at par with something like rclone. I did not test sshfs specifically but I am curious how long it takes to copy this amount of data using normal tool in your setup? For example, how long does a cp of so small small files take? (Just rough number is fine). Granted, Cloudron being node.js based is not going to be as fast as native tools, but given that both of them are just copying files sequentially, i don't expect that big a difference. (Like maybe Cloudron takes a minute or two more).

        1 Reply Last reply Reply Quote 0
        • girish
          girish Staff last edited by

          In the other thread, I saw that you were using encrypted backups. Is this the case when you tested this? Encryption requires a decent CPU as well, so a simple cp is not a fair comparison. I wonder if you can enable encryption on the remote disk somehow and not rely on Cloudron encryption?

          1 Reply Last reply Reply Quote 0
          • R
            Robin last edited by

            For the last release, I did some basic performance tests to get the backup speed to be at par with something like rclone. I did not test sshfs specifically but I am curious how long it takes to copy this amount of data using normal tool in your setup?

            My guess is that the problem is inherent to sshfs, and is not something specific to Cloudron (sshfs not being fast is pretty widely known/recognized). The only real reason I am reporting this is to perhaps suggest noting that it does not scale well in the docs, or something like that, because I don't think this gives a particularly good impression as-is.

            sshfs is very sensitive to latency, so any cases where you run it to a remote server on a different network, the slowdown will be very noticable.

            Using a tool more suited to this style of operation (like, say, rsync) will give better results, but it will still not scale well for large files (or large datasets, over say a few hundred GB of small files).

            In the other thread, I saw that you were using encrypted backups. Is this the case when you tested this? Encryption requires a decent CPU as well, so a simple cp is not a fair comparison. I wonder if you can enable encryption on the remote disk somehow and not rely on Cloudron encryption?

            Correct, encryption was enabled, but it wasn't really a factor. CPU was more or less idle while this was going on. Remote disk encryption isn't really an option in this case because I don't have physical control over the remote host, so I want to keep the data hidden 🙂

            1 Reply Last reply Reply Quote 1
            • First post
              Last post