Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Backup size growing

    Support
    backblaze backup
    6
    25
    1204
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • christiaan
      christiaan last edited by girish

      I'm backing up to Backblaze using rsync with a retention policy of effectively one backup. But the bucket keeps growing in size. One backup is about 2-3 GB but the bucket is now at 12.4 GB.

      Is this correct behaviour?

      nebulon girish 2 Replies Last reply Reply Quote 0
      • nebulon
        nebulon Staff @christiaan last edited by

        @christiaan is there anything unusual show in the backup logs, if you trigger a backup cleanup from the backups view of your Cloudron dashboard manually?

        1 Reply Last reply Reply Quote 1
        • girish
          girish Staff @christiaan last edited by

          @christiaan IIRC, there is a tricky setup in backblaze where "delete" doesn't actually delete but b2 keeps it around for a while and there is some setting to disable this behavior. I recall someone else ran into this as well (maybe it was @marcusquinn)

          marcusquinn 1 Reply Last reply Reply Quote 1
          • marcusquinn
            marcusquinn @girish last edited by

            @girish @christiaan Yes, that's part of their business model I guess.

            Naughty of them not to offer the setting when creating the Bucket, but I guess that's providing significant profits from the amount of times I've seen otherwise smart and prudent people caught out by it.

            It's under "Lifecycle Settings", you need to select "Keep only the last version of the file", given it is already versioned backups, there are no other versions you would need.

            I also recommend that if your host takes daily snapshots and you have a lot of data, you probably only need weekly offsite backups on your quietest day of the week.

            We're not here for a long time - but we are here for a good time :)
            Jersey/UK
            Work & Ecommerce Advice: https://brandlight.org
            Personal & Software Tips: https://marcusquinn.com

            christiaan 1 Reply Last reply Reply Quote 1
            • christiaan
              christiaan @marcusquinn last edited by christiaan

              @marcusquinn oh right. I did see that setting but I was following Cloudron instructions not to use it:
              https://docs.cloudron.io/backups/

              Screenshot 2021-06-29 at 20.09.32.png

              marcusquinn 1 Reply Last reply Reply Quote 0
              • marcusquinn
                marcusquinn @christiaan last edited by

                @christiaan Ahh, that's even more confusing then as Backblaze is using the same word; "lifecycle", to mean something different to Amazon's. Devil is in the detail on these platforms, where they mostly rely on lack of user calculation or reading for the most profitable features.

                We're not here for a long time - but we are here for a good time :)
                Jersey/UK
                Work & Ecommerce Advice: https://brandlight.org
                Personal & Software Tips: https://marcusquinn.com

                christiaan 1 Reply Last reply Reply Quote 1
                • christiaan
                  christiaan @marcusquinn last edited by

                  @marcusquinn ha, okay thanks.

                  I've also just clocked that I was quoting Amazon instructions instead Backblaze! Just to add to the general confusion, lol.

                  I turned on Backblaze's Lifecycle Settings to use 'Keep only the last version of the file' and the bucket size has reduced, thanks 👌🏼

                  So this won't corrupt my rsync files then?

                  marcusquinn D 2 Replies Last reply Reply Quote 1
                  • marcusquinn
                    marcusquinn @christiaan last edited by

                    @christiaan Nope, Cloudron handles the component parts that make up a whole Backup to restore from. You just don't need multiple copies of those parts.

                    Yeah, that should reduce your storage usage with Backblaze a lot.

                    You can try a test restore if you're not confident but I'm pretty confident from what you've explained.

                    As much as I wanted Rsync to work for us, I've moved to a weekly Tarball instead, relying instead on the daily host VPS snapshots as a low-resource cost version of the most recent data, with the Cloudron backups just as a secondary location backup for that redundancy.

                    We're not here for a long time - but we are here for a good time :)
                    Jersey/UK
                    Work & Ecommerce Advice: https://brandlight.org
                    Personal & Software Tips: https://marcusquinn.com

                    1 Reply Last reply Reply Quote 1
                    • D
                      dylightful @christiaan last edited by

                      @christiaan said in Backup size growing:

                      I turned on Backblaze's Lifecycle Settings to use 'Keep only the last version of the file' and the bucket size has reduced, thanks

                      Is this a permanent fix? Or does this just delete the occasional backup?

                      christiaan 1 Reply Last reply Reply Quote 0
                      • girish
                        girish Staff last edited by

                        @christiaan @marcusquinn thanks for note. I have updated the b2 docs accordingly - https://docs.cloudron.io/backups/#backblaze-b2

                        1 Reply Last reply Reply Quote 2
                        • christiaan
                          christiaan @dylightful last edited by

                          @dylightful permanent fix

                          @girish no prob, thanks

                          1 Reply Last reply Reply Quote 1
                          • christiaan
                            christiaan last edited by

                            Actually I'm not sure that was a permanent fix. It appears to be storing an additional copy each time there's an upgrade.

                            Screenshot 2021-07-17 at 13.46.18.png

                            jdaviescoates 1 Reply Last reply Reply Quote 0
                            • jdaviescoates
                              jdaviescoates @christiaan last edited by

                              @christiaan I think it's normal for those to be retained

                              I use Cloudron with Gandi & Hetzner

                              christiaan 1 Reply Last reply Reply Quote 1
                              • christiaan
                                christiaan @jdaviescoates last edited by

                                @jdaviescoates how do I remove them safely? Can I manually delete any of these folders without corrupting the backup?

                                Screenshot 2021-07-19 at 11.21.50.png

                                jdaviescoates girish 2 Replies Last reply Reply Quote 0
                                • jdaviescoates
                                  jdaviescoates @christiaan last edited by

                                  @christiaan I'm not sure, but I think you could probably delete all of them apart from the two at the bottom (the most up to date backup and the snapshots) but I wouldn't proceed without getting further clarity from @Staff 🙂

                                  I use Cloudron with Gandi & Hetzner

                                  1 Reply Last reply Reply Quote 1
                                  • girish
                                    girish Staff @christiaan last edited by girish

                                    @christiaan yes, you can safely delete any single timestamped directory safely. The snapshot directory should be left alone. If you click inside a timestamped directory and only find a single app_xx file, it means that backup was created as part of an app upgrade or backup. If you find many app_ files and a box_ file inside it, it was created as part of a Cloudron upgrade or backup.

                                    christiaan 1 Reply Last reply Reply Quote 2
                                    • christiaan
                                      christiaan @girish last edited by

                                      @girish Great, thanks. Would it be possible for you to automate this in Cloudron (i.e. deleting old versions)?

                                      jdaviescoates 1 Reply Last reply Reply Quote 0
                                      • jdaviescoates
                                        jdaviescoates @christiaan last edited by jdaviescoates

                                        @christiaan they would've been deleted eventually. I think maybe after 3 weeks.

                                        But yeah, would be nice for the to be a UI where they could be deleted too. I think that there might even be another post about that

                                        I use Cloudron with Gandi & Hetzner

                                        girish christiaan 2 Replies Last reply Reply Quote 2
                                        • girish
                                          girish Staff @jdaviescoates last edited by girish

                                          @jdaviescoates I am always wary about adding that feature because a compromised cloudron would then mean that backups can also be deleted. In many backup providers, you can create storage credentials with permissions so that objects cannot be deleted. Then again maybe just like auto-deletion, we can have a button to delete which may or may not work...

                                          christiaan 1 Reply Last reply Reply Quote 2
                                          • christiaan
                                            christiaan @jdaviescoates last edited by

                                            @jdaviescoates said in Backup size growing:

                                            @christiaan they would've been deleted eventually. I think maybe after 3 weeks.

                                            Oh right. There was one there that was a month old. Is there anywhere in Cloudron documentation that explains this?

                                            girish 1 Reply Last reply Reply Quote 0
                                            • girish
                                              girish Staff @christiaan last edited by

                                              @christiaan some info here - https://docs.cloudron.io/backups/#retention-policy

                                              christiaan 1 Reply Last reply Reply Quote 0
                                              • christiaan
                                                christiaan @girish last edited by

                                                @girish said in Backup size growing:

                                                @jdaviescoates I am always wary about adding that feature because a compromised cloudron would then mean that backups can also be deleted. In many backup providers, you can create storage credentials with permissions so that objects cannot be deleted. Then again maybe just like auto-deletion, we can have a button to delete which may or may not work...

                                                Why are they not simply deleted based on the retention policy? Is this for wiggle room if an upgrade goes wrong?

                                                My retention policy is 2 days, for instance, but one of these old version backups was a month old.

                                                1 Reply Last reply Reply Quote 0
                                                • christiaan
                                                  christiaan @girish last edited by

                                                  @girish said in Backup size growing:

                                                  @christiaan some info here - https://docs.cloudron.io/backups/#retention-policy

                                                  Thanks girish. This doesn't seem to cover the issue I am seeing though, which is that backups of old versions are not being deleted as per my retention policy.

                                                  girish 1 Reply Last reply Reply Quote 0
                                                  • girish
                                                    girish Staff @christiaan last edited by

                                                    @christiaan Correct me if I am wrong but i think the older ones are retained because of the previous b2 life cycle setting. Cloudron tries to cleanup backups only once. It won't go back and try to delete those old backups. As in, it won't delete files it thinks are already gone. Can you check if recent backups are indeed getting deleted? i.e the ones after you made the lifecycle setting change.

                                                    christiaan 1 Reply Last reply Reply Quote 2
                                                    • christiaan
                                                      christiaan @girish last edited by

                                                      @girish ah yes, that makes sense. And recent ones are being deleted, yes. Thanks again.

                                                      1 Reply Last reply Reply Quote 2
                                                      • First post
                                                        Last post
                                                      Powered by NodeBB