Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Discuss
  3. Using Object Storage for Cloudron Docker and Cloudron Backup?

Using Object Storage for Cloudron Docker and Cloudron Backup?

Scheduled Pinned Locked Moved Discuss
cloudronobject storagedockerbackup-storagestorage
20 Posts 6 Posters 2.7k Views 6 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • robiR robi

      @nebulon it's more about keeping those large docker images on s3 off the local disk as they just consume lots of space. Not about backup.

      girishG Offline
      girishG Offline
      girish
      Staff
      wrote on last edited by
      #7

      @robi sorry, I meant docker images in my earlier post. Currently, docker cannot run apps with images external to the server. They have to be downloaded locally to the server.

      robiR 1 Reply Last reply
      1
      • girishG girish

        @robi sorry, I meant docker images in my earlier post. Currently, docker cannot run apps with images external to the server. They have to be downloaded locally to the server.

        robiR Offline
        robiR Offline
        robi
        wrote on last edited by
        #8

        @girish Right, unless that is a local mount of a remote FS or s3 object store via rclone.

        It would be beneficial to not have to waste so much space from the OS as well as docker images plus apps data all locally.

        This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

        You wouldn't have to republish all apps with a new base image, right?

        Conscious tech

        scookeS 1 Reply Last reply
        0
        • robiR robi

          @girish Right, unless that is a local mount of a remote FS or s3 object store via rclone.

          It would be beneficial to not have to waste so much space from the OS as well as docker images plus apps data all locally.

          This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

          You wouldn't have to republish all apps with a new base image, right?

          scookeS Offline
          scookeS Offline
          scooke
          wrote on last edited by
          #9

          @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

          This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

          If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

          A life lived in fear is a life half-lived

          MooCloud_MattM 2 Replies Last reply
          0
          • scookeS scooke

            @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

            This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

            If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

            MooCloud_MattM Offline
            MooCloud_MattM Offline
            MooCloud_Matt
            wrote on last edited by
            #10

            Hello @scooke
            I will replay splitting your question/douth a part.

            • Cloudron already use just one container for all the MySQL database.
              What @robi is suggesting is to optimize how the docker image is created.
            • A monolithic or layer image can obtain the same, and it doesn't matter what control panel you use.
            • images built with layers will be a lot more space efficient because they share the same basic "image from" and they will not use space that is not requested.
              example: if you just use MySQL for a WordPress, why should you have MongoDB, pSQL, turn on your images?
              But if you add all the images that you would create using layers together they will use the same space as the current monolithic image approach.

            Matteo. R.
            Founder and Tech-Support Manager.
            MooCloud MSP
            Swiss Managed Service Provider

            1 Reply Last reply
            1
            • scookeS scooke

              @robi said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

              This also brings up the issue if the monolithic large base image vs small image layers of just what is needed for each app.

              If by this you mean Cloudron could have one MySQL image for all apps needing that, or one NGINX image for all apps needing proxying, etc. ... I'm not such a fan of that idea. I've tried that with what, portainer, and so much hassle. Even though it requires more room, I much prefer Cloudron putting everything an app needs into one image.

              MooCloud_MattM Offline
              MooCloud_MattM Offline
              MooCloud_Matt
              wrote on last edited by
              #11

              @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

              I much prefer Cloudron putting everything an app needs into one image.

              This is not advised because you will not need everything every time, which will use your space, increasing security issues, because, after all, it's harder to realize updates into a monolithic base image.

              Matteo. R.
              Founder and Tech-Support Manager.
              MooCloud MSP
              Swiss Managed Service Provider

              1 Reply Last reply
              1
              • MooCloud_MattM Offline
                MooCloud_MattM Offline
                MooCloud_Matt
                wrote on last edited by MooCloud_Matt
                #12

                This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                This will reduce by a reasonable amount of the space used by the cloudron base image.

                99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                Matteo. R.
                Founder and Tech-Support Manager.
                MooCloud MSP
                Swiss Managed Service Provider

                scookeS robiR 2 Replies Last reply
                1
                • MooCloud_MattM MooCloud_Matt

                  This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                  With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                  This will reduce by a reasonable amount of the space used by the cloudron base image.

                  99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                  And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                  scookeS Offline
                  scookeS Offline
                  scooke
                  wrote on last edited by
                  #13

                  @MooCloud_Matt I like the diagram. Did you make it?
                  Also, isn't this exactly what Cloudron does though?

                  A life lived in fear is a life half-lived

                  MooCloud_MattM 1 Reply Last reply
                  0
                  • scookeS scooke

                    @MooCloud_Matt I like the diagram. Did you make it?
                    Also, isn't this exactly what Cloudron does though?

                    MooCloud_MattM Offline
                    MooCloud_MattM Offline
                    MooCloud_Matt
                    wrote on last edited by
                    #14

                    @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                    Did you make it?

                    yes, I did =D

                    Also, isn't this exactly what Cloudron does though?

                    Nope, its not, cloudron includes everything, you always have:
                    Node 18.12.1
                    Go 1.19.3
                    Ruby 3.0
                    PHP 8.1
                    Python 3.10
                    Redis
                    Mysql
                    Mongo
                    pSQL

                    This for version 4.
                    If you want to keep compatibility with an older WordPress version because your plugin will not support 8.1.
                    you have to keep all it's included in cloudron base image v3

                    Matteo. R.
                    Founder and Tech-Support Manager.
                    MooCloud MSP
                    Swiss Managed Service Provider

                    scookeS 1 Reply Last reply
                    1
                    • MooCloud_MattM MooCloud_Matt

                      @scooke said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                      Did you make it?

                      yes, I did =D

                      Also, isn't this exactly what Cloudron does though?

                      Nope, its not, cloudron includes everything, you always have:
                      Node 18.12.1
                      Go 1.19.3
                      Ruby 3.0
                      PHP 8.1
                      Python 3.10
                      Redis
                      Mysql
                      Mongo
                      pSQL

                      This for version 4.
                      If you want to keep compatibility with an older WordPress version because your plugin will not support 8.1.
                      you have to keep all it's included in cloudron base image v3

                      scookeS Offline
                      scookeS Offline
                      scooke
                      wrote on last edited by
                      #15

                      @MooCloud_Matt I didn't realize this! So my WP app, my Vaultwarden, Matrix, Mastodon, Github, Grav, Jellyfin, Navidrome, Nextcloud, Vikunja, Umami, OpenVPN, LAMP app, etc. has ALL of those in the image that is installed when I click Install X app in the app store?? Why are all of those needed? I have 52 apps installed... so each of those apps has ALL of that software in its image?? Wow.

                      I assumed that, for example with the LAMP app, that the image would include Apache, MySQL, and PHP. No node, no Go, no Ruby, etc. This is quite eye-opening.

                      A life lived in fear is a life half-lived

                      MooCloud_MattM 1 Reply Last reply
                      1
                      • scookeS scooke

                        @MooCloud_Matt I didn't realize this! So my WP app, my Vaultwarden, Matrix, Mastodon, Github, Grav, Jellyfin, Navidrome, Nextcloud, Vikunja, Umami, OpenVPN, LAMP app, etc. has ALL of those in the image that is installed when I click Install X app in the app store?? Why are all of those needed? I have 52 apps installed... so each of those apps has ALL of that software in its image?? Wow.

                        I assumed that, for example with the LAMP app, that the image would include Apache, MySQL, and PHP. No node, no Go, no Ruby, etc. This is quite eye-opening.

                        MooCloud_MattM Offline
                        MooCloud_MattM Offline
                        MooCloud_Matt
                        wrote on last edited by
                        #16

                        @scooke
                        Not exactly; the image is stored once and used multiple times.
                        1 image multiple containers.
                        That means that you will not be impacted by a more optimized image because any way you use all its components.

                        But it will impact those who rent a 10 or 20 gb VPS because they will most likely use just part of that image, maybe just wordpres, or just a small nextcloud.
                        and they don't need ruby or node

                        Matteo. R.
                        Founder and Tech-Support Manager.
                        MooCloud MSP
                        Swiss Managed Service Provider

                        1 Reply Last reply
                        1
                        • MooCloud_MattM MooCloud_Matt

                          This is an example if a server was hosting just a single WordPress, if we can optimize the same cloudron base image, this is how many unused dependencies and packages are included that we don't use.

                          With just WordPress, we have to have the light base image + some debug tools + MySQL and the wordpress image itself that is base on apache + PHP (just the selected version that you are interested in) + wordpress core.

                          This will reduce by a reasonable amount of the space used by the cloudron base image.

                          99ba6ab6-50f6-4a5a-8ccf-205cc5ecb5af-image.png

                          And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                          robiR Offline
                          robiR Offline
                          robi
                          wrote on last edited by
                          #17

                          @MooCloud_Matt said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                          And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                          You can if you attach remote storage via things like rclone.

                          Conscious tech

                          MooCloud_MattM 1 Reply Last reply
                          0
                          • robiR robi

                            @MooCloud_Matt said in Using Object Storage for Cloudron Docker and Cloudron Backup?:

                            And to conclude, I think that you can't store in an easy and reliable way images on an s3 server, because images are stored just once, even if you start multiple containers.

                            You can if you attach remote storage via things like rclone.

                            MooCloud_MattM Offline
                            MooCloud_MattM Offline
                            MooCloud_Matt
                            wrote on last edited by
                            #18

                            @robi
                            yes, but then your image is used to run all your apps, which means that all those apps to run will have to get data from s3.

                            Remember that that image is not replicated anywhere else in the Filesystem, the container read their dependency from that file.
                            It would be like booting your server from s3, not a good idea, but that doesn't mean that storage footprint optimization is not possible in another way.

                            Matteo. R.
                            Founder and Tech-Support Manager.
                            MooCloud MSP
                            Swiss Managed Service Provider

                            robiR 1 Reply Last reply
                            0
                            • MooCloud_MattM MooCloud_Matt

                              @robi
                              yes, but then your image is used to run all your apps, which means that all those apps to run will have to get data from s3.

                              Remember that that image is not replicated anywhere else in the Filesystem, the container read their dependency from that file.
                              It would be like booting your server from s3, not a good idea, but that doesn't mean that storage footprint optimization is not possible in another way.

                              robiR Offline
                              robiR Offline
                              robi
                              wrote on last edited by
                              #19

                              @MooCloud_Matt agreed, yet once it's loaded into RAM, there isn't much access to the filesystem at that layer. "Streaming" apps from S3 will have an impact on startup times and performance, but saves a lot of space.

                              A hybrid mode could be where long term layers are cached in RAM, short term offloaded remotely and essential app data kept locally which CL backups take care of.

                              Conscious tech

                              MooCloud_MattM 1 Reply Last reply
                              0
                              • robiR robi

                                @MooCloud_Matt agreed, yet once it's loaded into RAM, there isn't much access to the filesystem at that layer. "Streaming" apps from S3 will have an impact on startup times and performance, but saves a lot of space.

                                A hybrid mode could be where long term layers are cached in RAM, short term offloaded remotely and essential app data kept locally which CL backups take care of.

                                MooCloud_MattM Offline
                                MooCloud_MattM Offline
                                MooCloud_Matt
                                wrote on last edited by
                                #20

                                @robi
                                On that, I don't have the data or experience to talk.
                                But 100% sure you can't use wasabi latency, I think smb or better NFS should be a better option, but I'm not an expert.
                                and Using so much ram for it, im not big fun about it, but I would have to try it before to have a real feedback

                                Matteo. R.
                                Founder and Tech-Support Manager.
                                MooCloud MSP
                                Swiss Managed Service Provider

                                1 Reply Last reply
                                0
                                Reply
                                • Reply as topic
                                Log in to reply
                                • Oldest to Newest
                                • Newest to Oldest
                                • Most Votes


                                  • Login

                                  • Don't have an account? Register

                                  • Login or register to search.
                                  • First post
                                    Last post
                                  0
                                  • Categories
                                  • Recent
                                  • Tags
                                  • Popular
                                  • Bookmarks
                                  • Search