Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Navigation

    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    SOLVED Are (encrypted) rsync backups not "incremental" on B2?

    Support
    backblaze backups rsync
    4
    16
    137
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • necrevistonnezr
      necrevistonnezr @robi last edited by

      @robi said in Are (encrypted) rsync backups not "incremental" on B2?:

      @necrevistonnezr oh that is great, and I want to set up something similar, more on that later.

      Are you aware that tar has an incremental mode?
      Then the local copy is just the tiny changes from the initial and the upload is also tiny.

      No, is that built-in into Cloudron's backup?

      Is odcrypt a feature of onedrive or rclone? What is the performance hit using it?

      odcrypt is just a random name I gave it. You first configure onedrive or google drive or whatever (https://rclone.org/docs/) and then configure encrpytion (https://rclone.org/crypt/). I haven't seen (albeit not checked, either) a performance hit.

      Have you been able to mount things like googledrive with the latest rclone?

      I can mount Onedrive encrypted without problem, so Google Drive should work as it's supported (https://rclone.org/commands/rclone_mount/)

      robi 1 Reply Last reply Reply Quote 0
      • girish
        girish Staff @necrevistonnezr last edited by

        @necrevistonnezr Ah ok, so it is sort of working as the code is designed. Currently, rsync is optimizing on uploads and only uploading incremental changes. But the final storage is not incremental, every backup (i.e timestamped directory in the remote) is a full complete standalone backup.

        necrevistonnezr 1 Reply Last reply Reply Quote 0
        • robi
          robi @necrevistonnezr last edited by

          @necrevistonnezr said

          No, is that built-in into Cloudron's backup?

          No, but if you test it and provide results, it may get added.

          I can mount Onedrive encrypted without problem, so Google Drive should work as it's supported (https://rclone.org/commands/rclone_mount/)

          Perfect, I have to get to this soon, thanks for the detail!

          necrevistonnezr 1 Reply Last reply Reply Quote 0
          • necrevistonnezr
            necrevistonnezr @girish last edited by necrevistonnezr

            @girish said in Are (encrypted) rsync backups not "incremental" on B2?:

            @necrevistonnezr Ah ok, so it is sort of working as the code is designed. Currently, rsync is optimizing on uploads and only uploading incremental changes. But the final storage is not incremental, every backup (i.e timestamped directory in the remote) is a full complete standalone backup.

            So that would result in huge amounts of space on the remote backup facility if you choose a certain retention, in my case (3 daily, 4 weekly, 6 monthly) something like 15 x 300 GB = 4,5 TB(!), correct?.

            It would be great if there was some deduplication done beforehand...? Or is using the --copy-destoption? (see some discussion here: https://www.linuxquestions.org/questions/linux-server-73/incremental-backups-with-rsync-link-dest-4175603276/)

            girish 1 Reply Last reply Reply Quote 1
            • necrevistonnezr
              necrevistonnezr @robi last edited by

              @robi BTW restic works with rclone, meaning you can use restic on anything that rclone can access (which is quite a lot, though restic already works with many cloud providers out of the box...

              robi 1 Reply Last reply Reply Quote 0
              • robi
                robi @necrevistonnezr last edited by

                @necrevistonnezr oh sweet, wasn't familiar with restic. And sure makes sense, if rclone can mount it, restic can backup to it.

                It sounds like @staff should take a look at restic+rclone for greatly enhancing backups in Cloudron with minimal work!

                We'd also get double free dedupe (once with restic and once on the target side with rclone) and minimal incremental backups, so efficient! ❤

                1 Reply Last reply Reply Quote 1
                • girish
                  girish Staff @necrevistonnezr last edited by

                  @necrevistonnezr said in Are (encrypted) rsync backups not "incremental" on B2?:

                  So that would result in huge amounts of space on the remote backup facility if you choose a certain retention, in my case (3 daily, 4 weekly, 6 monthly) something like 15 x 300 GB = 4,5 TB(!), correct?.

                  That's correct. There is no de-duplication on the backend.

                  necrevistonnezr 1 Reply Last reply Reply Quote 0
                  • necrevistonnezr
                    necrevistonnezr @girish last edited by

                    @girish
                    Thanks! You should seriously check out restic.

                    • The daily incremental backups I set up consume 200 MB if I changed 200 MBs over a day on a 300 GB repository.
                    • Takes 20 Minutes.
                    • Backups are encrypted.
                    • Many providers built in, the rest works via rclone. Backups are mountable for easy access to individual files.
                    marcusquinn 1 Reply Last reply Reply Quote 1
                    • marcusquinn
                      marcusquinn @necrevistonnezr last edited by

                      @necrevistonnezr said in Are (encrypted) rsync backups not "incremental" on B2?:

                      restic

                      Interesting, their sponsors platform looks interesting too: https://appscode.com/

                      1 Reply Last reply Reply Quote 0
                      • girish
                        girish Staff last edited by

                        For restic, can you upvote here - https://forum.cloudron.io/topic/1575/backup-improvements-restic-backend ? It should be possible to add a new restic backend like tgz/rsync. Though, we still have to figure out how we can "support" it. But that's a separate topic.

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post