Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Discuss
  3. Using Object Storage for Cloudron Docker and Cloudron Backup?

Using Object Storage for Cloudron Docker and Cloudron Backup?

Scheduled Pinned Locked Moved Discuss
cloudronobject storagedockerbackup-storagestorage
20 Posts 6 Posters 2.7k Views 6 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • girishG girish

      Docker has to be kept on disk, it doesn't support object storage. As for, Cloudron backup, you can configure this to be on any S3 compatible provider.

      robiR Offline
      robiR Offline
      robi
      wrote on last edited by
      #3

      @girish I think he meant docker images, not docker itself.

      Conscious tech

      1 Reply Last reply
      1
      • nebulonN Offline
        nebulonN Offline
        nebulon
        Staff
        wrote on last edited by
        #4

        If you were referring to docker images, then those are not even backed up.

        L robiR 2 Replies Last reply
        0
        • nebulonN nebulon

          If you were referring to docker images, then those are not even backed up.

          L Offline
          L Offline
          LoudLemur
          wrote on last edited by LoudLemur
          #5

          @nebulon Thanks for looking at this. The two large items I was looking at are the first two that appear if you go into your Cloudron's "System Information" and look at the Disk Usage chart at the top right. You can probably see that the first two components are quite large, relative to everything else. They are named "cloudron-backup" and "docker".

          By the way, any progress on supporting the deletion of particular backup files?

          1 Reply Last reply
          0
          • nebulonN nebulon

            If you were referring to docker images, then those are not even backed up.

            robiR Offline
            robiR Offline
            robi
            wrote on last edited by
            #6

            @nebulon it's more about keeping those large docker images on s3 off the local disk as they just consume lots of space. Not about backup.

            Conscious tech

            girishG 1 Reply Last reply
            1
            • robiR robi

              @nebulon it's more about keeping those large docker images on s3 off the local disk as they just consume lots of space. Not about backup.

              girishG Offline
              girishG Offline
              girish
              Staff
              wrote on last edited by
              #7

              @robi sorry, I meant docker images in my earlier post. Currently, docker cannot run apps with images external to the server. They have to be downloaded locally to the server.

              robiR 1 Reply Last reply
              1
              • girishG girish

                @robi sorry, I meant docker images in my earlier post. Currently, docker cannot run apps with images external to the server. They have to be downloaded locally to the server.

                robiR Offline
                robiR Offline
                robi
                wrote on last edited by
                #8

                @girish Right, unless that is a local mount of a remote FS or s3 object store via rclone.

                It would be beneficial to not have to waste so much space from the OS as well as docker images plus apps data all locally.

                This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                You wouldn't have to republish all apps with a new base image, right?

                Conscious tech

                scookeS 1 Reply Last reply
                0
                • robiR robi

                  @girish Right, unless that is a local mount of a remote FS or s3 object store via rclone.

                  It would be beneficial to not have to waste so much space from the OS as well as docker images plus apps data all locally.

                  This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                  You wouldn't have to republish all apps with a new base image, right?

                  scookeS Offline
                  scookeS Offline
                  scooke
                  wrote on last edited by
                  #9

                  @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                  This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                  If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

                  A life lived in fear is a life half-lived

                  MooCloud_MattM 2 Replies Last reply
                  0
                  • scookeS scooke

                    @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                    This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                    If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

                    MooCloud_MattM Offline
                    MooCloud_MattM Offline
                    MooCloud_Matt
                    wrote on last edited by
                    #10

                    Hello @scooke
                    I will replay splitting your question/douth a part.

                    • Cloudron already use just one container for all the MySQL database.
                      What @robi is suggesting is to optimize how the docker image is created.
                    • A monolithic or layer image can obtain the same, and it doesn't matter what control panel you use.
                    • images built with layers will be a lot more space efficient because they share the same basic "image from" and they will not use space that is not requested.
                      example: if you just use MySQL for a WordPress, why should you have MongoDB, pSQL, turn on your images?
                      But if you add all the images that you would create using layers together they will use the same space as the current monolithic image approach.

                    Matteo. R.
                    Founder and Tech-Support Manager.
                    MooCloud MSP
                    Swiss Managed Service Provider

                    1 Reply Last reply
                    1
                    • scookeS scooke

                      @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                      This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

                      If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

                      MooCloud_MattM Offline
                      MooCloud_MattM Offline
                      MooCloud_Matt
                      wrote on last edited by
                      #11

                      @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                      I much prefer Cloudron putting everything an app needs into one image.

                      This is not advised because you will not need everything every time, which will use your space, increasing security issues, because, after all, it's harder to realize updates into a monolithic base image.

                      Matteo. R.
                      Founder and Tech-Support Manager.
                      MooCloud MSP
                      Swiss Managed Service Provider

                      1 Reply Last reply
                      1
                      • MooCloud_MattM Offline
                        MooCloud_MattM Offline
                        MooCloud_Matt
                        wrote on last edited by MooCloud_Matt
                        #12

                        This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                        With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                        This will reduce by a reasonable amount of the space used by the cloudron base image.

                        99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                        And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                        Matteo. R.
                        Founder and Tech-Support Manager.
                        MooCloud MSP
                        Swiss Managed Service Provider

                        scookeS robiR 2 Replies Last reply
                        1
                        • MooCloud_MattM MooCloud_Matt

                          This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                          With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                          This will reduce by a reasonable amount of the space used by the cloudron base image.

                          99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                          And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                          scookeS Offline
                          scookeS Offline
                          scooke
                          wrote on last edited by
                          #13

                          @MooCloud_Matt I like the diagram. Did you make it?
                          Also, isn't this exactly what Cloudron does though?

                          A life lived in fear is a life half-lived

                          MooCloud_MattM 1 Reply Last reply
                          0
                          • scookeS scooke

                            @MooCloud_Matt I like the diagram. Did you make it?
                            Also, isn't this exactly what Cloudron does though?

                            MooCloud_MattM Offline
                            MooCloud_MattM Offline
                            MooCloud_Matt
                            wrote on last edited by
                            #14

                            @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                            Did you make it?

                            yes, I did =D

                            Also, isn't this exactly what Cloudron does though?

                            Nope, its not, cloudron includes everything, you always have:
                            Node 18.12.1
                            Go 1.19.3
                            Ruby 3.0
                            PHP 8.1
                            Python 3.10
                            Redis
                            Mysql
                            Mongo
                            pSQL

                            This for version 4.
                            If you want to keep compatibility with an older WordPress version because your plugin will not support 8.1.
                            you have to keep all it's included in cloudron base image v3

                            Matteo. R.
                            Founder and Tech-Support Manager.
                            MooCloud MSP
                            Swiss Managed Service Provider

                            scookeS 1 Reply Last reply
                            1
                            • MooCloud_MattM MooCloud_Matt

                              @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                              Did you make it?

                              yes, I did =D

                              Also, isn't this exactly what Cloudron does though?

                              Nope, its not, cloudron includes everything, you always have:
                              Node 18.12.1
                              Go 1.19.3
                              Ruby 3.0
                              PHP 8.1
                              Python 3.10
                              Redis
                              Mysql
                              Mongo
                              pSQL

                              This for version 4.
                              If you want to keep compatibility with an older WordPress version because your plugin will not support 8.1.
                              you have to keep all it's included in cloudron base image v3

                              scookeS Offline
                              scookeS Offline
                              scooke
                              wrote on last edited by
                              #15

                              @MooCloud_Matt I didn't realize this! So my WP app, my Vaultwarden, Matrix, Mastodon, Github, Grav, Jellyfin, Navidrome, Nextcloud, Vikunja, Umami, OpenVPN, LAMP app, etc. has ALL of those in the image that is installed when I click Install X app in the app store?? Why are all of those needed? I have 52 apps installed... so each of those apps has ALL of that software in its image?? Wow.

                              I assumed that, for example with the LAMP app, that the image would include Apache, MySQL, and PHP. No node, no Go, no Ruby, etc. This is quite eye-opening.

                              A life lived in fear is a life half-lived

                              MooCloud_MattM 1 Reply Last reply
                              1
                              • scookeS scooke

                                @MooCloud_Matt I didn't realize this! So my WP app, my Vaultwarden, Matrix, Mastodon, Github, Grav, Jellyfin, Navidrome, Nextcloud, Vikunja, Umami, OpenVPN, LAMP app, etc. has ALL of those in the image that is installed when I click Install X app in the app store?? Why are all of those needed? I have 52 apps installed... so each of those apps has ALL of that software in its image?? Wow.

                                I assumed that, for example with the LAMP app, that the image would include Apache, MySQL, and PHP. No node, no Go, no Ruby, etc. This is quite eye-opening.

                                MooCloud_MattM Offline
                                MooCloud_MattM Offline
                                MooCloud_Matt
                                wrote on last edited by
                                #16

                                @scooke
                                Not exactly; the image is stored once and used multiple times.
                                1 image multiple containers.
                                That means that you will not be impacted by a more optimized image because any way you use all its components.

                                But it will impact those who rent a 10 or 20 gb VPS because they will most likely use just part of that image, maybe just wordpres, or just a small nextcloud.
                                and they don't need ruby or node

                                Matteo. R.
                                Founder and Tech-Support Manager.
                                MooCloud MSP
                                Swiss Managed Service Provider

                                1 Reply Last reply
                                1
                                • MooCloud_MattM MooCloud_Matt

                                  This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                                  With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                                  This will reduce by a reasonable amount of the space used by the cloudron base image.

                                  99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                                  And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                                  robiR Offline
                                  robiR Offline
                                  robi
                                  wrote on last edited by
                                  #17

                                  @MooCloud_Matt said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                                  And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                                  You can if you attach remote storage via things like rclone.

                                  Conscious tech

                                  MooCloud_MattM 1 Reply Last reply
                                  0
                                  • robiR robi

                                    @MooCloud_Matt said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                                    And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                                    You can if you attach remote storage via things like rclone.

                                    MooCloud_MattM Offline
                                    MooCloud_MattM Offline
                                    MooCloud_Matt
                                    wrote on last edited by
                                    #18

                                    @robi
                                    yes, but then your image is used to run all your apps, which means that all those apps to run will have to get data from s3.

                                    Remember that that image is not replicated anywhere else in the Filesystem, the container read their dependency from that file.
                                    It would be like booting your server from s3, not a good idea, but that doesn't mean that storage footprint optimization is not possible in another way.

                                    Matteo. R.
                                    Founder and Tech-Support Manager.
                                    MooCloud MSP
                                    Swiss Managed Service Provider

                                    robiR 1 Reply Last reply
                                    0
                                    • MooCloud_MattM MooCloud_Matt

                                      @robi
                                      yes, but then your image is used to run all your apps, which means that all those apps to run will have to get data from s3.

                                      Remember that that image is not replicated anywhere else in the Filesystem, the container read their dependency from that file.
                                      It would be like booting your server from s3, not a good idea, but that doesn't mean that storage footprint optimization is not possible in another way.

                                      robiR Offline
                                      robiR Offline
                                      robi
                                      wrote on last edited by
                                      #19

                                      @MooCloud_Matt agreed, yet once it's loaded into RAM, there isn't much access to the filesystem at that layer. "Streaming" apps from S3 will have an impact on startup times and performance, but saves a lot of space.

                                      A hybrid mode could be where long term layers are cached in RAM, short term offloaded remotely and essential app data kept locally which CL backups take care of.

                                      Conscious tech

                                      MooCloud_MattM 1 Reply Last reply
                                      0
                                      • robiR robi

                                        @MooCloud_Matt agreed, yet once it's loaded into RAM, there isn't much access to the filesystem at that layer. "Streaming" apps from S3 will have an impact on startup times and performance, but saves a lot of space.

                                        A hybrid mode could be where long term layers are cached in RAM, short term offloaded remotely and essential app data kept locally which CL backups take care of.

                                        MooCloud_MattM Offline
                                        MooCloud_MattM Offline
                                        MooCloud_Matt
                                        wrote on last edited by
                                        #20

                                        @robi
                                        On that, I don't have the data or experience to talk.
                                        But 100% sure you can't use wasabi latency, I think smb or better NFS should be a better option, but I'm not an expert.
                                        and Using so much ram for it, im not big fun about it, but I would have to try it before to have a real feedback

                                        Matteo. R.
                                        Founder and Tech-Support Manager.
                                        MooCloud MSP
                                        Swiss Managed Service Provider

                                        1 Reply Last reply
                                        0
                                        Reply
                                        • Reply as topic
                                        Log in to reply
                                        • Oldest to Newest
                                        • Newest to Oldest
                                        • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Bookmarks
                                          • Search