Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Using Object Storage for Cloudron Docker and Cloudron Backup?

    Discuss
    cloudron object storage docker backup-storage storage
    6
    20
    184
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      LoudLemur last edited by

      These two are quite large and between them consume a large proportion of a Cloudron instance's storage. Large files lend themselves to object storage. Would it be a good idea and relatively simple to have Cloudron keep these two components on some dedicated, object storage? This would free up space for other uses on the instance.

      1 Reply Last reply Reply Quote 0
      • girish
        girish Staff last edited by

        Docker has to be kept on disk, it doesn't support object storage. As for, Cloudron backup, you can configure this to be on any S3 compatible provider.

        robi 1 Reply Last reply Reply Quote 1
        • robi
          robi @girish last edited by

          @girish I think he meant docker images, not docker itself.

          Life of Advanced Technology

          1 Reply Last reply Reply Quote 1
          • nebulon
            nebulon Staff last edited by

            If you were referring to docker images, then those are not even backed up.

            L robi 2 Replies Last reply Reply Quote 0
            • L
              LoudLemur @nebulon last edited by LoudLemur

              @nebulon Thanks for looking at this. The two large items I was looking at are the first two that appear if you go into your Cloudron's "System Information" and look at the Disk Usage chart at the top right. You can probably see that the first two components are quite large, relative to everything else. They are named "cloudron-backup" and "docker".

              By the way, any progress on supporting the deletion of particular backup files?

              1 Reply Last reply Reply Quote 0
              • robi
                robi @nebulon last edited by

                @nebulon it's more about keeping those large docker images on s3 off the local disk as they just consume lots of space. Not about backup.

                Life of Advanced Technology

                girish 1 Reply Last reply Reply Quote 1
                • girish
                  girish Staff @robi last edited by

                  @robi sorry, I meant docker images in my earlier post. Currently, docker cannot run apps with images external to the server. They have to be downloaded locally to the server.

                  robi 1 Reply Last reply Reply Quote 1
                  • robi
                    robi @girish last edited by

                    @girish Right, unless that is a local mount of a remote FS or s3 object store via rclone.

                    It would be beneficial to not have to waste so much space from the OS as well as docker images plus apps data all locally.

                    This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                    You wouldn't have to republish all apps with a new base image, right?

                    Life of Advanced Technology

                    scooke 1 Reply Last reply Reply Quote 0
                    • scooke
                      scooke @robi last edited by

                      @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                      This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                      If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

                      A life lived in fear is a life half-lived

                      MooCloud_Matt 2 Replies Last reply Reply Quote 0
                      • MooCloud_Matt
                        MooCloud_Matt @scooke last edited by

                        Hello @scooke
                        I will replay splitting your question/douth a part.

                        • Cloudron already use just one container for all the MySQL database.
                          What @robi is suggesting is to optimize how the docker image is created.
                        • A monolithic or layer image can obtain the same, and it doesn't matter what control panel you use.
                        • images built with layers will be a lot more space efficient because they share the same basic "image from" and they will not use space that is not requested.
                          example: if you just use MySQL for a WordPress, why should you have MongoDB, pSQL, turn on your images?
                          But if you add all the images that you would create using layers together they will use the same space as the current monolithic image approach.

                        Matteo. R.
                        Founder and Tech-Support Manager.
                        MooCloud MSP
                        Swiss Managed Service Provider

                        1 Reply Last reply Reply Quote 1
                        • MooCloud_Matt
                          MooCloud_Matt @scooke last edited by

                          @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                          I much prefer Cloudron putting everything an app needs into one image.

                          This is not advised because you will not need everything every time, which will use your space, increasing security issues, because, after all, it's harder to realize updates into a monolithic base image.

                          Matteo. R.
                          Founder and Tech-Support Manager.
                          MooCloud MSP
                          Swiss Managed Service Provider

                          1 Reply Last reply Reply Quote 1
                          • MooCloud_Matt
                            MooCloud_Matt last edited by MooCloud_Matt

                            This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                            With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                            This will reduce by a reasonable amount of the space used by the cloudron base image.

                            99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                            And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                            Matteo. R.
                            Founder and Tech-Support Manager.
                            MooCloud MSP
                            Swiss Managed Service Provider

                            scooke robi 2 Replies Last reply Reply Quote 1
                            • scooke
                              scooke @MooCloud_Matt last edited by

                              @MooCloud_Matt I like the diagram. Did you make it?
                              Also, isn't this exactly what Cloudron does though?

                              A life lived in fear is a life half-lived

                              MooCloud_Matt 1 Reply Last reply Reply Quote 0
                              • MooCloud_Matt
                                MooCloud_Matt @scooke last edited by

                                @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                                Did you make it?

                                yes, I did =D

                                Also, isn't this exactly what Cloudron does though?

                                Nope, its not, cloudron includes everything, you always have:
                                Node 18.12.1
                                Go 1.19.3
                                Ruby 3.0
                                PHP 8.1
                                Python 3.10
                                Redis
                                Mysql
                                Mongo
                                pSQL

                                This for version 4.
                                If you want to keep compatibility with an older WordPress version because your plugin will not support 8.1.
                                you have to keep all it's included in cloudron base image v3

                                Matteo. R.
                                Founder and Tech-Support Manager.
                                MooCloud MSP
                                Swiss Managed Service Provider

                                scooke 1 Reply Last reply Reply Quote 1
                                • scooke
                                  scooke @MooCloud_Matt last edited by

                                  @MooCloud_Matt I didn't realize this! So my WP app, my Vaultwarden, Matrix, Mastodon, Github, Grav, Jellyfin, Navidrome, Nextcloud, Vikunja, Umami, OpenVPN, LAMP app, etc. has ALL of those in the image that is installed when I click Install X app in the app store?? Why are all of those needed? I have 52 apps installed... so each of those apps has ALL of that software in its image?? Wow.

                                  I assumed that, for example with the LAMP app, that the image would include Apache, MySQL, and PHP. No node, no Go, no Ruby, etc. This is quite eye-opening.

                                  A life lived in fear is a life half-lived

                                  MooCloud_Matt 1 Reply Last reply Reply Quote 1
                                  • MooCloud_Matt
                                    MooCloud_Matt @scooke last edited by

                                    @scooke
                                    Not exactly; the image is stored once and used multiple times.
                                    1 image multiple containers.
                                    That means that you will not be impacted by a more optimized image because any way you use all its components.

                                    But it will impact those who rent a 10 or 20 gb VPS because they will most likely use just part of that image, maybe just wordpres, or just a small nextcloud.
                                    and they don't need ruby or node

                                    Matteo. R.
                                    Founder and Tech-Support Manager.
                                    MooCloud MSP
                                    Swiss Managed Service Provider

                                    1 Reply Last reply Reply Quote 1
                                    • robi
                                      robi @MooCloud_Matt last edited by

                                      @MooCloud_Matt said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                                      And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                                      You can if you attach remote storage via things like rclone.

                                      Life of Advanced Technology

                                      MooCloud_Matt 1 Reply Last reply Reply Quote 0
                                      • MooCloud_Matt
                                        MooCloud_Matt @robi last edited by

                                        @robi
                                        yes, but then your image is used to run all your apps, which means that all those apps to run will have to get data from s3.

                                        Remember that that image is not replicated anywhere else in the Filesystem, the container read their dependency from that file.
                                        It would be like booting your server from s3, not a good idea, but that doesn't mean that storage footprint optimization is not possible in another way.

                                        Matteo. R.
                                        Founder and Tech-Support Manager.
                                        MooCloud MSP
                                        Swiss Managed Service Provider

                                        robi 1 Reply Last reply Reply Quote 0
                                        • robi
                                          robi @MooCloud_Matt last edited by

                                          @MooCloud_Matt agreed, yet once it's loaded into RAM, there isn't much access to the filesystem at that layer. "Streaming" apps from S3 will have an impact on startup times and performance, but saves a lot of space.

                                          A hybrid mode could be where long term layers are cached in RAM, short term offloaded remotely and essential app data kept locally which CL backups take care of.

                                          Life of Advanced Technology

                                          MooCloud_Matt 1 Reply Last reply Reply Quote 0
                                          • MooCloud_Matt
                                            MooCloud_Matt @robi last edited by

                                            @robi
                                            On that, I don't have the data or experience to talk.
                                            But 100% sure you can't use wasabi latency, I think smb or better NFS should be a better option, but I'm not an expert.
                                            and Using so much ram for it, im not big fun about it, but I would have to try it before to have a real feedback

                                            Matteo. R.
                                            Founder and Tech-Support Manager.
                                            MooCloud MSP
                                            Swiss Managed Service Provider

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Powered by NodeBB