Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    [Backups] Ability to add multiple storage provider/location

    Feature Requests
    12
    37
    1415
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • edapm
      edapm @girish last edited by

      @girish Yes, I would be good to say have a Minio bucket to which to push backups to on your friends' Cloudrons, and be able to have a backup on each Cloudron. (See diagram below)

      4273a466-385d-4ecc-be95-82421765049a-image.png

      *Small square is Minio bucket

      1 Reply Last reply Reply Quote 3
      • marcusquinn
        marcusquinn last edited by

        It might be more CPU/bandwidth friendly if one Rsync process sends the backup to location A, as it does right now, and a second initiates the copy from B to C, and if desired a D, E, F location too if people wanted to go crazy with backup locations.

        From another angle, it might be that we want one entirely different backup method to one place (could be unencrypted rsync), and another to another (perhaps encrypted tarball).

        Would very much like this though, especially as cyber attacks are only growing as more and more economic value is online, and on other people's servers.

        We're not here for a long time - but we are here for a good time :)
        Jersey/UK
        Work & Ecommerce Advice: https://brandlight.org
        Personal & Software Tips: https://marcusquinn.com

        girish 1 Reply Last reply Reply Quote 5
        • girish
          girish Staff @marcusquinn last edited by

          @marcusquinn yes, we will try to do this for the release after. I think it's important to support 3-2-1 style backups - atleast the 3-2 part more easily.

          MooCloud_Matt 1 Reply Last reply Reply Quote 5
          • MooCloud_Matt
            MooCloud_Matt @girish last edited by

            A good way to handle the 3-2-1 for cloudron can be to replicate what Proxmox has done, delegate the replication to the other software(Proxmox Backup Server) installed on the destination backup server, this is for 2 reasons: storage servers/vps often have low resources but those are good enough to enable replication and they are really cheap, probably more expensive then Wasabi, but Wasabi is slow and really not efficient in cost for cold Rsync backup.

            Matteo. R.
            Founder and Tech-Support Manager.
            MooCloud MSP
            Swiss Managed Service Provider

            MooCloud_Matt 1 Reply Last reply Reply Quote 3
            • MooCloud_Matt
              MooCloud_Matt @MooCloud_Matt last edited by MooCloud_Matt

              I was looking around for a way to solve this issue without using a complicated setup like Ceph (like we did, and we would love to get out of it but that's another story).
              I found Restic, it could be used instead of rsync, and it supports S3 but also their own rest server that can run on the destination server of the backup and with their cli, you can create a copy of one of the snapshots without involving the production server, and store it on an HDD offline or on the others providers.

              https://github.com/restic
              https://github.com/restic/rest-server

              Matteo. R.
              Founder and Tech-Support Manager.
              MooCloud MSP
              Swiss Managed Service Provider

              jdaviescoates 1 Reply Last reply Reply Quote 1
              • jdaviescoates
                jdaviescoates @MooCloud_Matt last edited by

                @moocloud_matt said in [Backups] Ability to add multiple storage provider/location:

                Restic

                There's a bunch of discussion about that here:

                https://forum.cloudron.io/post/2466

                Sounds like both @necrevistonnezr and @fbartels have some experience with it

                I use Cloudron with Gandi & Hetzner

                fbartels 1 Reply Last reply Reply Quote 1
                • fbartels
                  fbartels App Dev @jdaviescoates last edited by

                  @jdaviescoates yes, restic is quite nice. I am however not using their server backend, but rather push backups to a s3 target.

                  I am however not sure how restic would solve the 3rd (offsite) part. I kind of achieve this by mirroring s3 buckets, but this is something I can already do when Cloudron is already writing stuff to s3.

                  MooCloud_Matt 1 Reply Last reply Reply Quote 0
                  • MooCloud_Matt
                    MooCloud_Matt @fbartels last edited by

                    @fbartels
                    you have a command/API that can copy a snapshot to a new location.
                    (i just had time to check their documentation and a basic install)

                    Matteo. R.
                    Founder and Tech-Support Manager.
                    MooCloud MSP
                    Swiss Managed Service Provider

                    fbartels 1 Reply Last reply Reply Quote 0
                    • fbartels
                      fbartels App Dev @MooCloud_Matt last edited by fbartels

                      @moocloud_matt ah, true. This however means that the copy needs to go through a local client (download & upload) and apparently also reencrypts data (with the potential of deduplication not working).

                      https://restic.readthedocs.io/en/latest/045_working_with_repos.html#copying-snapshots-between-repositories

                      Its been ages since I last looked into restic, as it "just worked" for me. The thing to highlight is that apparently there has been a successful handover in maintainership in the past, as the original author hasn't done any work himself in quite a while.

                      Another upside is its portability due to being written in golang.

                      Edit: if one wants to look into restic, https://autorestic.vercel.app/ is a nice wrapper to simplify setup and handling. My old systems still use bits of bash for that.

                      MooCloud_Matt 1 Reply Last reply Reply Quote 0
                      • MooCloud_Matt
                        MooCloud_Matt @fbartels last edited by

                        @fbartels said in [Backups] Ability to add multiple storage provider/location:

                        local client (download & upload)

                        True but is Go, and a Storage Server/VPS have enough resources to have both server and client on it to replicate to an cold s3 storage.

                        @fbartels said in [Backups] Ability to add multiple storage provider/location:

                        Another upside is its portability due to being written in golang.

                        We also need to take into consideration that setup a golang script is easy and can be done by almost anybody in the cloudron community and tanks to the Rest API, cloudron can manage the server super easily, instead if it will be selected a nodejs or py script it can be a lot more difficult to setup.
                        I think that docker should be excluded as a tool to distribute the destination server for the backup, because many VPS provider of Storage Server offers OpenVZ and not KVM, we would not have that issue but many the will.

                        Matteo. R.
                        Founder and Tech-Support Manager.
                        MooCloud MSP
                        Swiss Managed Service Provider

                        1 Reply Last reply Reply Quote 0
                        • robi
                          robi last edited by

                          Don't forget about rclone.org

                          Life of Advanced Technology

                          MooCloud_Matt 1 Reply Last reply Reply Quote 0
                          • MooCloud_Matt
                            MooCloud_Matt @robi last edited by

                            @robi
                            I actually don't like that the 3-2-1 is managed by the main server, because if that is compromised you will have compromised also your backup.

                            I think that: if cloudron wants to offer a better backup solution should have a 3° party software/node be in charge of the replication for the 2-1.
                            This will protect the server from any ransomware or if your server is compromised.

                            Matteo. R.
                            Founder and Tech-Support Manager.
                            MooCloud MSP
                            Swiss Managed Service Provider

                            robi 1 Reply Last reply Reply Quote 0
                            • marcusquinn
                              marcusquinn last edited by

                              Borrowing an answer from StackOverflow that may work:

                              • Minio Cloudron instance: using the command mc mirror on a cron job.

                              If that works, it could just be a case of documenting and maybe a GUI to make it user-friendly @girish ?

                              We're not here for a long time - but we are here for a good time :)
                              Jersey/UK
                              Work & Ecommerce Advice: https://brandlight.org
                              Personal & Software Tips: https://marcusquinn.com

                              MooCloud_Matt 1 Reply Last reply Reply Quote 0
                              • MooCloud_Matt
                                MooCloud_Matt @marcusquinn last edited by

                                @marcusquinn
                                a full install of cloudron is too many resources w8st for many storage servers, we speak of old CPU (many of our storage servers have Haswell xeon) or just 1 vCore (time4vps) , and often without the support of docker.

                                Matteo. R.
                                Founder and Tech-Support Manager.
                                MooCloud MSP
                                Swiss Managed Service Provider

                                1 Reply Last reply Reply Quote 1
                                • robi
                                  robi @MooCloud_Matt last edited by

                                  @moocloud_matt said in [Backups] Ability to add multiple storage provider/location:

                                  I actually don't like that the 3-2-1 is managed by the main server, because if that is compromised you will have compromised also your backup.

                                  That's the problem with traditional backups.

                                  Next gen way of thinking about backups is simply having a much more resilient storage system. For example, when your data is sprinkled across 8 places and you only need 5 to restore any file/object. There are some very clever and efficient algorithms for this m of n approach which removes the need for 3x replication.

                                  Minio can do this, and as a community we can pool resources to have 20+ places and only need 7 or so to be available at any one time. Maybe even start a coop. 😄

                                  Life of Advanced Technology

                                  neurokrish MooCloud_Matt marcusquinn 3 Replies Last reply Reply Quote 5
                                  • neurokrish
                                    neurokrish @robi last edited by

                                    @robi great idea. +1 for m of n approach with minio!

                                    1 Reply Last reply Reply Quote 1
                                    • MooCloud_Matt
                                      MooCloud_Matt @robi last edited by

                                      @robi said in [Backups] Ability to add multiple storage provider/location:

                                      For example, when your data is sprinkled across 8 places and you only need 5 to restore any file/object.

                                      True, that's why we are using ceph, but it's not efficient (storage speaking) to protect the files we need to use snapshot or versioning in ceph too, because if the access of the bucket is compromised on the cloudron side all files even if they are slit into multiple nodes can still be deleted/encrypted, so that made the all advantage of using Software Define Storage/ Distributed storage = to a normal NAS offer by the datacenter over NFS.

                                      I really would like to analyze better what push proxmox to build a dedicated Client for their Storage Server. And what I have understood un till now is that they want maximal protection made easy which means that the ssh key used by their hypervisor server is not able to access the 2 and 1 copy of the backup.

                                      I really don't care about what software/ stack Cloudron will use, I just want to get out of Ceph for the backup and use a better setup that is not less safe.

                                      Matteo. R.
                                      Founder and Tech-Support Manager.
                                      MooCloud MSP
                                      Swiss Managed Service Provider

                                      robi 1 Reply Last reply Reply Quote 1
                                      • marcusquinn
                                        marcusquinn @robi last edited by

                                        @robi Kinda what IPFS is?

                                        • https://forum.cloudron.io/topic/6280/ipfs-desktop-cluster-peer-to-peer-hypermedia-file-sharing

                                        We're not here for a long time - but we are here for a good time :)
                                        Jersey/UK
                                        Work & Ecommerce Advice: https://brandlight.org
                                        Personal & Software Tips: https://marcusquinn.com

                                        1 Reply Last reply Reply Quote 0
                                        • robi
                                          robi @MooCloud_Matt last edited by

                                          @moocloud_matt proper object storage will have versioning and dedup too.

                                          If you have source compromise you have other issues, however aspects of the push to store can be flipped in a pull to archive manner provided you have a mechanism to detect compromise and stop the pull or avoid overwrites.

                                          Life of Advanced Technology

                                          MooCloud_Matt 1 Reply Last reply Reply Quote 0
                                          • MooCloud_Matt
                                            MooCloud_Matt @robi last edited by MooCloud_Matt

                                            @robi

                                            @robi said in [Backups] Ability to add multiple storage provider/location:

                                            will have versioning and dedup too

                                            True, but dedup is a "raid" equivalent feature but scaled to multi-node.
                                            And versioning is not efficient if combined with backup tools, because most of them don't take advantage of s3 versioning API.

                                            So we are back on the idea that the main server if it's is compromised you want your backup to be safe, so many ransomware those days.

                                            Yes, that's possible to do it with S3 but is not easy and requires a lot of coding from the cloudron side and is just 1 protocol, no sftp or nfs or samba, if you don't want to use a proxy s3 to nfs or like we use Ceph to smd.

                                            Supporting 1 protocol is not what backups are in cloudron, you see more and more protocol support over time, I just help add SMB3 (with encrypt) to that list.

                                            And I think that is amazing for cloudron users that are not geeks or professionals and want to keep their local Synology or qnap NAS as a backup.

                                            A solution that would fit users like many that are commenting on this post is not enough. The idea I hope is to find a solution that can scale from a classic rsync stile to a 3-2-1 systems like proxmox backup or Borg.

                                            Matteo. R.
                                            Founder and Tech-Support Manager.
                                            MooCloud MSP
                                            Swiss Managed Service Provider

                                            robi 1 Reply Last reply Reply Quote 0
                                            • robi
                                              robi @MooCloud_Matt last edited by

                                              @moocloud_matt for your Ceph replacment you should look at Cleversafe from IBM. They also have some partners that make it even better on the user side. Komprise comes to mind.

                                              For Cloudron, a nice rclone integration would handle most of those options nicely.

                                              Life of Advanced Technology

                                              1 Reply Last reply Reply Quote 1
                                              • d19dotca
                                                d19dotca last edited by

                                                Definitely having a need for this now as I look to further improve my DR plan. What I’m hoping to achieve… faster local backups to an external block storage disk daily (possibly even multiple times given criticality of email for some users), and perhaps once a week or every 2-5 days would be an external backup to something like Wasabi to avoid any complete disasters in a single datacentre.

                                                --
                                                Dustin Dauncey
                                                www.d19.ca

                                                1 Reply Last reply Reply Quote 2
                                                • girish
                                                  girish Staff last edited by

                                                  Yes, this is high on our list. Just finishing up the backup async rewrite! Almost there with 7.2.

                                                  P 1 Reply Last reply Reply Quote 5
                                                  • P
                                                    p44 translator @girish last edited by

                                                    @girish Hello Girish, any updates on this feature? Thank's 🙏

                                                    girish 1 Reply Last reply Reply Quote 1
                                                    • girish
                                                      girish Staff @p44 last edited by

                                                      @p44 probably the next release, this release is already a bit packed (and delayed)

                                                      P 2 Replies Last reply Reply Quote 4
                                                      • P
                                                        p44 translator @girish last edited by

                                                        @girish Thank's, take care

                                                        1 Reply Last reply Reply Quote 1
                                                        • Referenced by  d19dotca d19dotca 
                                                        • P
                                                          p44 translator @girish last edited by

                                                          @girish Hello Girish, any news on this feature? Many thank's 🙏

                                                          1 Reply Last reply Reply Quote 2
                                                          • Referenced by  girish girish 
                                                          • J
                                                            jayonrails translator last edited by

                                                            I can only agree with the previous speaker. I would be very happy about an implementation. For me it is not foreseeable how much effort it can be.

                                                            1 Reply Last reply Reply Quote 3
                                                            • marcusquinn
                                                              marcusquinn last edited by

                                                              Might be a case of packaging the Rclone Web GUI as an app, then people can duplicate their main S3 backups anywhere else as a secondary routine.

                                                              • https://forum.cloudron.io/topic/7159/rclone-web-gui

                                                              We're not here for a long time - but we are here for a good time :)
                                                              Jersey/UK
                                                              Work & Ecommerce Advice: https://brandlight.org
                                                              Personal & Software Tips: https://marcusquinn.com

                                                              P 1 Reply Last reply Reply Quote 4
                                                              • P
                                                                p44 translator @marcusquinn last edited by

                                                                @marcusquinn This could be an alternative solution but I hope that a second backup can be configured directly in Cloudron.

                                                                1 Reply Last reply Reply Quote 2
                                                                • First post
                                                                  Last post
                                                                Powered by NodeBB