Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Docker registry

    App Wishlist
    9
    65
    4884
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • robi
      robi last edited by

      Amazing progress, thank you @mario

      Now to figure out how to install this and inch another step in packaging a custom app..

      Life of Gratitude.
      Life of Advanced Technology

      1 Reply Last reply Reply Quote 0
      • fbartels
        fbartels App Dev @mario last edited by fbartels

        @mario would you mind me making an attempt of adding https://github.com/Joxit/docker-registry-ui/ to your app? Or would you rather do it yourself, or choose a different ui?

        Another solution could be the reg cli utility. A simple docker binary that can also expose a ui.

        @robi after you installed to Cloudron cli (should be on a Linux machine with docker installed) you just need to run cloudron build && cloudron install and then follow the prompts.

        mario 1 Reply Last reply Reply Quote 2
        • mario
          mario App Dev @fbartels last edited by mario

          @fbartels honestly, I'd prefer UI as a separate app and would attempt Portus - possibly together with you. What are your thoughts?

          An alternative would indeed be, if people prefer, to have Registry + UI together -> but in any case I'd strongly prefer Portus to anything else.

          fbartels 1 Reply Last reply Reply Quote 1
          • girish
            girish Staff @mario last edited by

            @mario Oh wow, this is awesome. I had no idea one could run a registry this way. I thought one has to make some use of the docker addon! This way is so much simpler and nicer.

            I forked the code to https://git.cloudron.io/cloudron/docker-registry-app/ and gave you permissions. It just worked (tm). Do you think you can put in a LICENSE file and keep developing there? It's a holiday for thanksgiving here, but I will look into this soonish.

            mario 1 Reply Last reply Reply Quote 2
            • fbartels
              fbartels App Dev @mario last edited by

              @mario TIL portus does not implement it's own registry, but simply uses the official one.

              Yes, having them separate can have it's benefits. Would need to refresh my knowledge in regards to portus first before I know if I could be of much help.

              mario 1 Reply Last reply Reply Quote 0
              • mario
                mario App Dev @girish last edited by

                @girish happy Thanksgiving! 🙂

                Thanks for the fork, will see what I can do in the coming days 😉

                1 Reply Last reply Reply Quote 0
                • girish
                  girish Staff @mehdi last edited by

                  @mehdi said in Docker registry:

                  Combined with the authProxy of Cloudron6, we could disable the htpasswd auth of the app and since it uses basic auth it should work with the LDAP users !

                  I gave this a try and this worked great! https://git.cloudron.io/cloudron/docker-registry-app/-/commit/547e3b30b0d9038d9fe73416a7df7b3d32f265ec

                  mario 1 Reply Last reply Reply Quote 2
                  • mario
                    mario App Dev @fbartels last edited by

                    @fbartels said in Docker registry:

                    @mario TIL portus does not implement it's own registry, but simply uses the official one.

                    Yes, having them separate can have it's benefits. Would need to refresh my knowledge in regards to portus first before I know if I could be of much help.

                    Indeed 🙂 It basically takes advantage of the official registry support for token-auth, giving you a nice UI, permissions, etc.

                    When you get a moment to check it out, let's talk! 🙂

                    1 Reply Last reply Reply Quote 1
                    • mario
                      mario App Dev @girish last edited by mario

                      @girish said in Docker registry:

                      @mehdi said in Docker registry:

                      Combined with the authProxy of Cloudron6, we could disable the htpasswd auth of the app and since it uses basic auth it should work with the LDAP users !

                      I gave this a try and this worked great! https://git.cloudron.io/cloudron/docker-registry-app/-/commit/547e3b30b0d9038d9fe73416a7df7b3d32f265ec

                      The only problem here is that this would not work for me - we basically only have admins on Cloudron itself, and this would limit Registry access to them alone.

                      Edit: this is because we do auth via Azure AD/SAML for pretty much everything in the company.

                      girish 1 Reply Last reply Reply Quote 0
                      • girish
                        girish Staff @mario last edited by

                        @mario said in Docker registry:

                        The only problem here is that this would not work for me

                        Good point. I forgot to add the optionalSso flag to manifest. With that flag, you can install the app without Cloudron Directory integration (like you do with other apps) and then we can have the default admin/admin setup that you have when LDAP is disabled. Would that work?

                        mario 1 Reply Last reply Reply Quote 1
                        • mario
                          mario App Dev @girish last edited by

                          @girish yes, though there's a bug in your commit 😛 So let's make sure we fix that too.

                          girish 1 Reply Last reply Reply Quote 0
                          • girish
                            girish Staff @mario last edited by

                            @mario Ha ha, possibly. I only hacked it up quickly and checked if proxyAuth code in 6.0 will work before I make the release.

                            mario 1 Reply Last reply Reply Quote 1
                            • mario
                              mario App Dev @girish last edited by

                              @girish enjoy your holiday and let me know when you're back around next week so we can take this further 🙂

                              1 Reply Last reply Reply Quote 0
                              • mehdi
                                mehdi App Dev last edited by

                                @girish great ! I was 90% sure it would work, I'm glad I got it right 😛

                                About interfaces, Portus indeed looks really great. However, I really don't see how it would work as a separate app. I really think it makes sense to bundle them together.

                                1 Reply Last reply Reply Quote 3
                                • girish
                                  girish Staff last edited by

                                  Is Portus still developed? It seems it has seen no commits since Mar 25 2020?

                                  1 Reply Last reply Reply Quote 0
                                  • atridad
                                    atridad App Dev last edited by

                                    I am both excited about this and confused about where it is at. My endgame is using GitLab to manage containers, but I need to point it at a registry. Would this ultimately work? And is there a way to have auth go through gitlab for this?

                                    I type things and sometimes those things end up in your browser. 🏳️‍🌈🇮🇷🇨🇦

                                    mario 1 Reply Last reply Reply Quote 1
                                    • mario
                                      mario App Dev @atridad last edited by

                                      @atrilahiji it's currently working as a stand-alone registry via basic auth powered by htpasswd file. It'll also support Cloudron SSO shortly, after that I'll work on making it work with GitLab.

                                      robi 1 Reply Last reply Reply Quote 4
                                      • robi
                                        robi @mario last edited by

                                        @mario how's it going?

                                        Life of Gratitude.
                                        Life of Advanced Technology

                                        mario 1 Reply Last reply Reply Quote 2
                                        • mario
                                          mario App Dev @robi last edited by

                                          @robi I have managed to integrate the registry with GitLab.

                                          @girish where are we at with making proper MR for AuthProxy + making SSO optional? Then I can document GitLab integration, you can write some tests and off we go!

                                          girish 1 Reply Last reply Reply Quote 4
                                          • girish
                                            girish Staff @mario last edited by

                                            @mario Fantastic news. So, all we need is docs to make it work with GitLab registry or does it need any packages changes to gitlab app or docker registry app ?

                                            mario 1 Reply Last reply Reply Quote 3
                                            • mario
                                              mario App Dev @girish last edited by

                                              @girish doesn't seem like it'll need package changes, documentation will be enough.

                                              mario 1 Reply Last reply Reply Quote 1
                                              • mario
                                                mario App Dev @mario last edited by mario

                                                I take that back, I did add some package changes. Had no time to test, but things seem to be working ok from the initial glimpse at it:

                                                https://git.cloudron.io/cloudron/docker-registry-app/-/merge_requests/1

                                                Please test and report back @girish and others 🙂

                                                1 Reply Last reply Reply Quote 2
                                                • robi
                                                  robi last edited by

                                                  Might be useful to add auto deletion of old images:
                                                  https://github.com/jeffstephens/retention-manager

                                                  Life of Gratitude.
                                                  Life of Advanced Technology

                                                  mario 1 Reply Last reply Reply Quote 0
                                                  • mario
                                                    mario App Dev @robi last edited by

                                                    @robi GitLab does that for me 😛 Maybe a separate app? 🙂

                                                    girish robi 2 Replies Last reply Reply Quote 0
                                                    • girish
                                                      girish Staff @mario last edited by

                                                      @mario Just looking into this now.

                                                      Wondering, what is the best way forward. The app has no UI, but can have a login screen (via proxyAuth). So, when they login, they see a blank screen. Not ideal. Does it make sense to bundle any of the docker uis like https://github.com/Joxit/docker-registry-ui/ ? Seems quite easy to do. I can look into it.

                                                      mario 1 Reply Last reply Reply Quote 2
                                                      • robi
                                                        robi @mario last edited by

                                                        @mario that's great, but the standalone private registry app that's coming may need it and as @girish pointed out a simple UI.

                                                        Also not everyone is interested in the ruby laden GitLab and all it's complexity. 🙂

                                                        Life of Gratitude.
                                                        Life of Advanced Technology

                                                        1 Reply Last reply Reply Quote 0
                                                        • mario
                                                          mario App Dev @girish last edited by

                                                          @girish depends on what the community needs. I'm more than happy to have a separate registry + other things as separate apps for those who need it.

                                                          If I needed to pick the best registry solution with UI and everything else that's well maintained and suitable for Cloudron, I'd probably look at Quay which supports LDAP auth.

                                                          https://github.com/quay/quay

                                                          1 Reply Last reply Reply Quote 1
                                                          • mehdi
                                                            mehdi App Dev last edited by

                                                            I am 100% in favor of bundling a simple UI together with the registry. Even if one does not need it and wants to use the gitlab UI, there's basically nothing to lose besides a few kB of storage ^^

                                                            1 Reply Last reply Reply Quote 1
                                                            • jimcavoli
                                                              jimcavoli App Dev last edited by

                                                              Yeah, Quay and Harbor are definitely the big players in this space. Very similar products - harbor is CNCF graduated and Quay is upstream for the corresponding Red Hat product. Either (or both) would be good UI adds.

                                                              mario 1 Reply Last reply Reply Quote 1
                                                              • girish
                                                                girish Staff last edited by

                                                                Last I checked harbor was impractical to package (as in way too much effort, it's really geared for the k8s crowd). Quay is a good option, but let me get this basic docker registry out first, I am almost there.

                                                                1 Reply Last reply Reply Quote 3
                                                                • mario
                                                                  mario App Dev @jimcavoli last edited by

                                                                  @jimcavoli Quay afaik implements the protocol as well, so no need for registry separately.

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • girish
                                                                    girish Staff last edited by

                                                                    So strange, I am getting a "invalid checksum digest format" whenever I push now to this registry. Has anyone seen such an error before?

                                                                    The push refers to repository [xxx.xxx.xxx/cloudron/base]
                                                                    fcdfeda3e242: Layer already exists 
                                                                    0ea3bde29271: Layer already exists 
                                                                    d75ccb14b8b6: Layer already exists 
                                                                    74b4389a43ab: Layer already exists 
                                                                    5f38ae1e1a63: Layer already exists 
                                                                    3479c151673d: Layer already exists 
                                                                    7a307b866f25: Layer already exists 
                                                                    ce3a66c20e17: Layer already exists 
                                                                    7197b970ebb9: Layer already exists 
                                                                    16542a8fc3be: Layer already exists 
                                                                    6597da2e2e52: Layer already exists 
                                                                    977183d4e999: Layer already exists 
                                                                    c8be1b8f4d60: Layer already exists 
                                                                    invalid checksum digest format
                                                                    
                                                                    mario 1 Reply Last reply Reply Quote 0
                                                                    • mario
                                                                      mario App Dev @girish last edited by

                                                                      @girish local filesystem?

                                                                      girish 1 Reply Last reply Reply Quote 0
                                                                      • girish
                                                                        girish Staff @mario last edited by girish

                                                                        @mario Yes, with the local storage. I wonder if it's something to do with the proxy auth. I am trying it without auth now.

                                                                        edit: indeed, something to do with the proxy auth. It works fine without proxy auth. Debugging.

                                                                        mehdi 1 Reply Last reply Reply Quote 0
                                                                        • mehdi
                                                                          mehdi App Dev @girish last edited by

                                                                          @girish Are you on 6.1 ? Maybe your 2FA implementation broke something with the basic auth ?

                                                                          girish 1 Reply Last reply Reply Quote 0
                                                                          • girish
                                                                            girish Staff @mehdi last edited by

                                                                            @mehdi yeah, i had that in mind and tried with 6.0 as well. fails the same. I am pretty sure this worked when I tested it back then, so I must have broke something !

                                                                            mehdi 1 Reply Last reply Reply Quote 0
                                                                            • mehdi
                                                                              mehdi App Dev @girish last edited by

                                                                              @girish You can try with an app-password, or try another Basic Auth ProxyAuth app, like Transmission (with an android app or a browser extension)

                                                                              1 Reply Last reply Reply Quote 0
                                                                              • girish
                                                                                girish Staff last edited by

                                                                                What I am seeing is that docker doesn't send any authorization header at all. The issue is very similar to https://stackoverflow.com/questions/55516317/docker-login-not-passing-basic-authentication-headers-to-nginx . I can curl just fine.

                                                                                1 Reply Last reply Reply Quote 0
                                                                                • girish
                                                                                  girish Staff last edited by

                                                                                  It seems that v2 registry auth does not use the basic bearer based authentication at all. https://docs.docker.com/registry/recipes/nginx/ is possibly obsolete, but I am trying to setup a registry from scratch now to double check.

                                                                                  mario mehdi 2 Replies Last reply Reply Quote 2
                                                                                  • mario
                                                                                    mario App Dev @girish last edited by

                                                                                    @girish it definitely can, that's how GitLab etc integration works.

                                                                                    girish 1 Reply Last reply Reply Quote 0
                                                                                    • mehdi
                                                                                      mehdi App Dev @girish last edited by

                                                                                      @girish Their doc indeed appears to be outdated. Different pages seem to indicate different things ...

                                                                                      1 Reply Last reply Reply Quote 0
                                                                                      • girish
                                                                                        girish Staff @mario last edited by girish

                                                                                        @mario thanks! i needed such a confident statement to help me keep looking further 🙂

                                                                                        I managed to get it to work. The issue is that proxyAuth on an auth fail redirects to the login page. But the docker registry wants it to return a 401 with a www-authenticate header. The header also causes issues with browsers since it starts popping up the login dialog.

                                                                                        In essence, even though the basic auth works, proxyAuth is not compatible. I thought about adding an flag to the manifest to have a different behavior but then again I don't like the current approach where we just install this registry and land on an empty page (any page even some static html with instructions would be better).

                                                                                        I ended up packaging it together the docker registry UI and a small LDAP server (from https://git.cloudron.io/cloudron/cloudron-serve). I haven't pushed the changes since they are not working entirely. But it's what I am working on in parallel with getting 6.1 out.

                                                                                        fbartels mehdi 2 Replies Last reply Reply Quote 4
                                                                                        • fbartels
                                                                                          fbartels App Dev @girish last edited by

                                                                                          @girish said in Docker registry:

                                                                                          I ended up packaging it together the docker registry UI and a small LDAP server

                                                                                          That sounds intriguing. What role does the ldap server serve? Just for auth against the registry ui?

                                                                                          girish 1 Reply Last reply Reply Quote 0
                                                                                          • mehdi
                                                                                            mehdi App Dev @girish last edited by

                                                                                            @girish I think the best approach would be to do a bit of user-agent parsing magic... Yeah, it would be quite specific for this use-case, but 🤷

                                                                                            girish 1 Reply Last reply Reply Quote 0
                                                                                            • girish
                                                                                              girish Staff @fbartels last edited by

                                                                                              @fbartels said in Docker registry:

                                                                                              That sounds intriguing. What role does the ldap server serve? Just for auth against the registry ui?

                                                                                              Yes, pretty much. It's just a proxy that redirects to login page and auths against LDAP. The code itself is very small, just ~100 lines or so.

                                                                                              1 Reply Last reply Reply Quote 0
                                                                                              • girish
                                                                                                girish Staff @mehdi last edited by

                                                                                                @mehdi Right, I considered UA string hack but I think dropping users in a blank page is a bit rough. So, my first step was to do the UA testing with nginx in the app itself. But, that brought the dreaded browser auth modal dialog which I really dislike. It's the main reason I ended up making proxyAuth in the first place 😉 So.. I ended up making a node server.

                                                                                                mehdi 1 Reply Last reply Reply Quote 0
                                                                                                • mehdi
                                                                                                  mehdi App Dev @girish last edited by

                                                                                                  @girish No, I mean, after testing you could keep the proxyAuth, but do a test on the proxyAuth that could show the page for browsers, and send the expected 401 for docker client. Then we could have the best of both worlds : integration with platform LDAP, a simple registry UI, and working CLI.

                                                                                                  girish 1 Reply Last reply Reply Quote 2
                                                                                                  • girish
                                                                                                    girish Staff @mehdi last edited by

                                                                                                    @mehdi Ah, understood you better now. I am actually ok to add this hack in proxy auth code. We will still need some nginx/apache in the app code though to serve the registry UI (which is just static html).

                                                                                                    Suddenly, I am tempted to abandon my node server because I am struggling to make this proxy middleware work. It seems to have some bug with PATCH requests which docker registry uses.

                                                                                                    1 Reply Last reply Reply Quote 0
                                                                                                    • girish
                                                                                                      girish Staff last edited by

                                                                                                      I have published this app as unstable now. It also has an integrated UI. I have only very mildly tested it, so do not use it in production. I have created an app category for this, please report any issues there.

                                                                                                      1 Reply Last reply Reply Quote 4
                                                                                                      • L
                                                                                                        LoudLemur @robi last edited by

                                                                                                        @robi Thanks,

                                                                                                        Quay is a Free alternative to DockerHub. Hopefully, Cloudron makes good use of it... ?

                                                                                                        robi 1 Reply Last reply Reply Quote 0
                                                                                                        • robi
                                                                                                          robi @LoudLemur last edited by

                                                                                                          @LoudLemur You can already use it by pointing your Cloudron to it.

                                                                                                          Life of Gratitude.
                                                                                                          Life of Advanced Technology

                                                                                                          1 Reply Last reply Reply Quote 0
                                                                                                          • First post
                                                                                                            Last post
                                                                                                          Powered by NodeBB