Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Feature Request: Backblaze B2 As Backup Target

    Feature Requests
    backups feature-request
    5
    22
    663
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • technotame
      technotame last edited by girish

      I'd like to request Backblaze B2 as a an option for backups. It's API is not S3 compatible unfortunately, but it is a cheaper option than S3.

      Thanks.

      1 Reply Last reply Reply Quote 10
      • girish
        girish Staff last edited by

        This is in our todo list.

        1 Reply Last reply Reply Quote 3
        • M
          moonmeister last edited by

          Would love to see this.

          1 Reply Last reply Reply Quote 1
          • M
            moonmeister last edited by

            Backblaze just announced an S3 compatible API for B2. Obviously users can set this up manually but hopefully this makes it faster to include a native integration as well!

            https://www.backblaze.com/blog/backblaze-b2-s3-compatible-api/

            1 Reply Last reply Reply Quote 5
            • girish
              girish Staff last edited by

              @moonmeister Yup, incredible announcement. They also lacked server side copies (which Cloudron requires)! I will be testing it out.

              1 Reply Last reply Reply Quote 3
              • girish
                girish Staff last edited by

                FWIW, you can probably already test this out with "S3 Compatible Provider" in the backups drop-down. If anyone manages to test it out, please let us know here.

                1 Reply Last reply Reply Quote 3
                • A
                  adrw last edited by

                  Running a full backup test now, will report back results but I'm very excited!

                  1 Reply Last reply Reply Quote 1
                  • A
                    adrw last edited by

                    Backup is still running, uploaded >77GB in 7 hours. Would highly recommend TAR backup format since with rsync it blows through the limited Class C Transactions with the Remote Copy and individual file uploads which would end up being pretty costly.

                    1 Reply Last reply Reply Quote 2
                    • girish
                      girish Staff last edited by

                      This will be part of 5.4

                      A 1 Reply Last reply Reply Quote 3
                      • A
                        adrw @girish last edited by

                        @girish Awesome! This is such exciting new! Thank you for prioritizing it and getting it into the next release.

                        I'm curious...

                        1. Will it be using the native B2 API or with their S3 compatible API?
                        2. Will it be optimized to support rsync backups and not just tar/zip?

                        Again, huge kudos for baking in B2 support!

                        marcusquinn 1 Reply Last reply Reply Quote 1
                        • girish
                          girish Staff last edited by

                          @adrw It will use the S3 compatible API. Both rsync and zip options will be available.

                          1 Reply Last reply Reply Quote 0
                          • marcusquinn
                            marcusquinn @adrw last edited by marcusquinn

                            @adrw Tested using the S3 compatibility setup and it works perfectly with rsync and tarballs, and the rsync actually doesn't cost anything significant so is still worth doing.

                            Very happy with the backup systems and both Backblaze B2 and Wasabi for speed and value. Great work team Cloudron - and hope that's some confident happiness for everyone wondering if either are worth trying.

                            We're not here for a long time - but we are here for a good time :)
                            Jersey/UK
                            Work & Ecommerce Advice: https://brandlight.org
                            Personal & Software Tips: https://marcusquinn.com

                            A 1 Reply Last reply Reply Quote 1
                            • A
                              adrw @marcusquinn last edited by

                              @marcusquinn Great to hear! Curious your experience with B2 rsync. I have a fairly large backup (~250GB) and it seems to blast through the Class C transactions very fast to the point of making rsync multiple times more expensive than tar zip backup. I’m giving it another shot based on your experience, hopefully the increased cost is only on first run.

                              @girish Is there a way I can increase the backup timeout? For my large (~250GB) backup it keeps timing out while uploading ~8mbps to B2 and making 65-70% progress. If the timeout can be configurable or increased it looks like it would succeed. Thanks!

                              marcusquinn 1 Reply Last reply Reply Quote 1
                              • girish
                                girish Staff last edited by

                                @adrw I am working on some fixed for the next release (as part of https://git.cloudron.io/cloudron/box/-/issues/691). Some settings like copy concurrency, upload concurrency, total backup timeout etc seem to be very installation specific. So, we will expose these in the UI.

                                A 1 Reply Last reply Reply Quote 2
                                • A
                                  adrw @girish last edited by

                                  @girish Great to hear and thanks for prioritizing this. For my own sake, when do you expect those settings will be exposed? I haven't had a successful backup in 3 days since some additional data was uploaded to a Nextcloud instance so I'm starting to sweat a bit. Both tar and rsync fail with the same timeout error after running for hours. If it can land in the next week or two then I'll probably stick it out until then. Otherwise I'm going to revert the data migration so that backups can start again. Many thanks again!

                                  A 1 Reply Last reply Reply Quote 0
                                  • marcusquinn
                                    marcusquinn @adrw last edited by

                                    @adrw If ingress is too pricey for your volumes with Backblaze then Wasabi is very competitive and doesn't charge for any ingress up or down.

                                    We're not here for a long time - but we are here for a good time :)
                                    Jersey/UK
                                    Work & Ecommerce Advice: https://brandlight.org
                                    Personal & Software Tips: https://marcusquinn.com

                                    A 1 Reply Last reply Reply Quote 0
                                    • A
                                      adrw @marcusquinn last edited by

                                      @marcusquinn Thanks for the recommendation! It only seems to be a bit pricey when using the rsync backup mode. For tar it works great. Both seem to timing out so I'm hoping @girish exposing some more knobs to turn will allow improvement of backup speed.

                                      1 Reply Last reply Reply Quote 1
                                      • A
                                        adrw @adrw last edited by

                                        @girish Good news, rsync backups are now succeeding since the diffs are small and a few failed ones were able to build on each other so no rush on the configuration. Looking forward to testing it out once it lands! Thanks again!

                                        1 Reply Last reply Reply Quote 1
                                        • girish
                                          girish Staff last edited by

                                          @adrw I have scheduled configuring the timing issues for 5.5 in any case. We have a couple of others facing similar issues.

                                          1 Reply Last reply Reply Quote 1
                                          • marcusquinn
                                            marcusquinn last edited by

                                            Update on using rsynch to Backblaze B2:

                                            On one Cloudron, I have 23 Apps installed, most are just default installs a few migrated websites.

                                            98GB of backup storage with lifecycle retention of last 2 versions. 1.1 million files.

                                            Backblaze B2 ingress seems to be costing about the same as the storage, last month was $4.95.

                                            The Billing page is shit too as they use the US date-format, the only country in the world with that weirdness and you'd think developers would know better. Maybe their measuring ingress in Pounds, Ounzes, Cups, Quarts, Feet & Inches? šŸ˜‚

                                            Conclusion being, Backblaze B2 will be cheaper if you're using tarball backups or for up to about 150GB rsync.

                                            Wasabi will be cheaper if you want rsync for incremental backups for more than about 150GB.

                                            Personally I'm leaning towards sticking with rsync for incremental and changing that target to Wasabi as storage needs grow.

                                            Absolute ideal would be to have both; tarball to B2 and rsync to Wasabi - or even have 2 x B2 accounts.

                                            That's all in addition to provider snapshots — so 3 separate physical, jurisdictional and provider locations. Could probably do more with Synchthing as well but that might be overdoing it.

                                            Can't think of a more comprehensive, risk-spread replication and lower cost setup than all that unless anyone has experience otherwise?

                                            We're not here for a long time - but we are here for a good time :)
                                            Jersey/UK
                                            Work & Ecommerce Advice: https://brandlight.org
                                            Personal & Software Tips: https://marcusquinn.com

                                            A 1 Reply Last reply Reply Quote 2
                                            • girish
                                              girish Staff last edited by

                                              Wasabi prices are kind of incredible, IMO and I have not seen cheaper options. B2 also has the aspect that download costs money. So if you want to restore 150GB, this costs $1.5.

                                              Only option that is comparable is probably Hetzner storage box. 1TB is around $9. This solution is only practical if your server is somewhere near their storage box.

                                              (Ignoring the cheapest option of buying a external hard disk and attaching it to your physical server šŸ™‚ I bought a 2TB disk for just $55. The disk will pay for itself in 8 months of cloud storage)

                                              1 Reply Last reply Reply Quote 2
                                              • A
                                                adrw @marcusquinn last edited by

                                                @marcusquinn Great summary! I've similarly found that ingress transaction costs are more than my monthly storage costs with rsync option and tar zip backups keep timing out.

                                                @girish Thanks for adding the additional advanced backup configuration in 5.5! I've tried increasing it for my backups and am excited to see if it helps.

                                                1 Reply Last reply Reply Quote 1
                                                • First post
                                                  Last post
                                                Powered by NodeBB