Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Docker registry

    App Wishlist
    9
    65
    8576
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • mario
      mario App Dev last edited by girish

      https://github.com/docker/distribution-library-image

      This would enable us to tie it in with Gitlab.

      Lonkle mario 2 Replies Last reply Reply Quote 4
      • Lonkle
        Lonkle @mario last edited by

        @mario To keep and maintain our own private Docker hubs?

        1 Reply Last reply Reply Quote 0
        • robi
          robi last edited by

          Adding Quay
          https://github.com/quay/quay
          Project Quay builds, stores, and distributes your container images.

          High-level features include:

          • Docker Registry Protocol v2
          • Docker Manifest Schema v2.1, v2.2
          • AppC Image Discovery via on-demand transcoding
          • Image Squashing via on-demand transcoding
          • Authentication provided by LDAP, Keystone, OIDC, Google, and GitHub
          • ACLs, team management, and auditability logs
          • Geo-replicated storage provided by local filesystems, S3, GCS, Swift, and Ceph
          • Continuous Integration integrated with GitHub, Bitbucket, GitLab, and git
          • Security Vulnerability Analysis via Clair
          • Swagger-compliant HTTP API

          Life of Advanced Technology

          L 1 Reply Last reply Reply Quote 5
          • mario
            mario App Dev @mario last edited by

            I got the registry working, need to get it integrated with GitLab now.

            girish Lonkle 2 Replies Last reply Reply Quote 3
            • girish
              girish Staff @mario last edited by

              @mario Did you get the registry working as a Cloudron package or outside Cloudron?

              mario 1 Reply Last reply Reply Quote 0
              • Lonkle
                Lonkle @mario last edited by

                @mario What he said ^^^ 😂

                1 Reply Last reply Reply Quote 0
                • robi
                  robi last edited by

                  anyone check his gitlab repo?

                  Life of Advanced Technology

                  Lonkle 1 Reply Last reply Reply Quote 0
                  • Lonkle
                    Lonkle @robi last edited by

                    @robi I don't know it. 😅

                    1 Reply Last reply Reply Quote 0
                    • mario
                      mario App Dev @girish last edited by mario

                      @girish as a Cloudron package. I'll push it to gitlab once I clean it up, but it's useless to me without the GitLab integration and that part seems tricky.

                      mario 1 Reply Last reply Reply Quote 2
                      • mario
                        mario App Dev @mario last edited by

                        @girish
                        https://git.cloudron.io/mario/docker-registry

                        mario girish 2 Replies Last reply Reply Quote 3
                        • mario
                          mario App Dev @mario last edited by

                          default user:

                          admin/admin

                          mehdi 1 Reply Last reply Reply Quote 0
                          • mehdi
                            mehdi App Dev @mario last edited by

                            @mario This looks good ! Combined with the authProxy of Cloudron6, we could disable the htpasswd auth of the app and since it uses basic auth it should work with the LDAP users !

                            We would just have to slap a basic web interface on that and it would be perfect 😄

                            mario girish 2 Replies Last reply Reply Quote 1
                            • mario
                              mario App Dev @mehdi last edited by

                              @mehdi I'm updating it further now with redis support etc.

                              mario 1 Reply Last reply Reply Quote 1
                              • mario
                                mario App Dev @mario last edited by

                                Updated and working.

                                fbartels 1 Reply Last reply Reply Quote 3
                                • robi
                                  robi last edited by

                                  Amazing progress, thank you @mario

                                  Now to figure out how to install this and inch another step in packaging a custom app..

                                  Life of Advanced Technology

                                  1 Reply Last reply Reply Quote 0
                                  • fbartels
                                    fbartels App Dev @mario last edited by fbartels

                                    @mario would you mind me making an attempt of adding https://github.com/Joxit/docker-registry-ui/ to your app? Or would you rather do it yourself, or choose a different ui?

                                    Another solution could be the reg cli utility. A simple docker binary that can also expose a ui.

                                    @robi after you installed to Cloudron cli (should be on a Linux machine with docker installed) you just need to run cloudron build && cloudron install and then follow the prompts.

                                    mario 1 Reply Last reply Reply Quote 2
                                    • mario
                                      mario App Dev @fbartels last edited by mario

                                      @fbartels honestly, I'd prefer UI as a separate app and would attempt Portus - possibly together with you. What are your thoughts?

                                      An alternative would indeed be, if people prefer, to have Registry + UI together -> but in any case I'd strongly prefer Portus to anything else.

                                      fbartels 1 Reply Last reply Reply Quote 1
                                      • girish
                                        girish Staff @mario last edited by

                                        @mario Oh wow, this is awesome. I had no idea one could run a registry this way. I thought one has to make some use of the docker addon! This way is so much simpler and nicer.

                                        I forked the code to https://git.cloudron.io/cloudron/docker-registry-app/ and gave you permissions. It just worked (tm). Do you think you can put in a LICENSE file and keep developing there? It's a holiday for thanksgiving here, but I will look into this soonish.

                                        mario 1 Reply Last reply Reply Quote 2
                                        • fbartels
                                          fbartels App Dev @mario last edited by

                                          @mario TIL portus does not implement it's own registry, but simply uses the official one.

                                          Yes, having them separate can have it's benefits. Would need to refresh my knowledge in regards to portus first before I know if I could be of much help.

                                          mario 1 Reply Last reply Reply Quote 0
                                          • mario
                                            mario App Dev @girish last edited by

                                            @girish happy Thanksgiving! 🙂

                                            Thanks for the fork, will see what I can do in the coming days 😉

                                            1 Reply Last reply Reply Quote 0
                                            • girish
                                              girish Staff @mehdi last edited by

                                              @mehdi said in Docker registry:

                                              Combined with the authProxy of Cloudron6, we could disable the htpasswd auth of the app and since it uses basic auth it should work with the LDAP users !

                                              I gave this a try and this worked great! https://git.cloudron.io/cloudron/docker-registry-app/-/commit/547e3b30b0d9038d9fe73416a7df7b3d32f265ec

                                              mario 1 Reply Last reply Reply Quote 2
                                              • mario
                                                mario App Dev @fbartels last edited by

                                                @fbartels said in Docker registry:

                                                @mario TIL portus does not implement it's own registry, but simply uses the official one.

                                                Yes, having them separate can have it's benefits. Would need to refresh my knowledge in regards to portus first before I know if I could be of much help.

                                                Indeed 🙂 It basically takes advantage of the official registry support for token-auth, giving you a nice UI, permissions, etc.

                                                When you get a moment to check it out, let's talk! 🙂

                                                1 Reply Last reply Reply Quote 1
                                                • mario
                                                  mario App Dev @girish last edited by mario

                                                  @girish said in Docker registry:

                                                  @mehdi said in Docker registry:

                                                  Combined with the authProxy of Cloudron6, we could disable the htpasswd auth of the app and since it uses basic auth it should work with the LDAP users !

                                                  I gave this a try and this worked great! https://git.cloudron.io/cloudron/docker-registry-app/-/commit/547e3b30b0d9038d9fe73416a7df7b3d32f265ec

                                                  The only problem here is that this would not work for me - we basically only have admins on Cloudron itself, and this would limit Registry access to them alone.

                                                  Edit: this is because we do auth via Azure AD/SAML for pretty much everything in the company.

                                                  girish 1 Reply Last reply Reply Quote 0
                                                  • girish
                                                    girish Staff @mario last edited by

                                                    @mario said in Docker registry:

                                                    The only problem here is that this would not work for me

                                                    Good point. I forgot to add the optionalSso flag to manifest. With that flag, you can install the app without Cloudron Directory integration (like you do with other apps) and then we can have the default admin/admin setup that you have when LDAP is disabled. Would that work?

                                                    mario 1 Reply Last reply Reply Quote 1
                                                    • mario
                                                      mario App Dev @girish last edited by

                                                      @girish yes, though there's a bug in your commit 😛 So let's make sure we fix that too.

                                                      girish 1 Reply Last reply Reply Quote 0
                                                      • girish
                                                        girish Staff @mario last edited by

                                                        @mario Ha ha, possibly. I only hacked it up quickly and checked if proxyAuth code in 6.0 will work before I make the release.

                                                        mario 1 Reply Last reply Reply Quote 1
                                                        • mario
                                                          mario App Dev @girish last edited by

                                                          @girish enjoy your holiday and let me know when you're back around next week so we can take this further 🙂

                                                          1 Reply Last reply Reply Quote 0
                                                          • mehdi
                                                            mehdi App Dev last edited by

                                                            @girish great ! I was 90% sure it would work, I'm glad I got it right 😛

                                                            About interfaces, Portus indeed looks really great. However, I really don't see how it would work as a separate app. I really think it makes sense to bundle them together.

                                                            1 Reply Last reply Reply Quote 3
                                                            • girish
                                                              girish Staff last edited by

                                                              Is Portus still developed? It seems it has seen no commits since Mar 25 2020?

                                                              1 Reply Last reply Reply Quote 0
                                                              • ?
                                                                A Former User last edited by

                                                                I am both excited about this and confused about where it is at. My endgame is using GitLab to manage containers, but I need to point it at a registry. Would this ultimately work? And is there a way to have auth go through gitlab for this?

                                                                mario 1 Reply Last reply Reply Quote 1
                                                                • mario
                                                                  mario App Dev @Guest last edited by

                                                                  @atrilahiji it's currently working as a stand-alone registry via basic auth powered by htpasswd file. It'll also support Cloudron SSO shortly, after that I'll work on making it work with GitLab.

                                                                  robi 1 Reply Last reply Reply Quote 3
                                                                  • robi
                                                                    robi @mario last edited by

                                                                    @mario how's it going?

                                                                    Life of Advanced Technology

                                                                    mario 1 Reply Last reply Reply Quote 2
                                                                    • mario
                                                                      mario App Dev @robi last edited by

                                                                      @robi I have managed to integrate the registry with GitLab.

                                                                      @girish where are we at with making proper MR for AuthProxy + making SSO optional? Then I can document GitLab integration, you can write some tests and off we go!

                                                                      girish 1 Reply Last reply Reply Quote 3
                                                                      • girish
                                                                        girish Staff @mario last edited by

                                                                        @mario Fantastic news. So, all we need is docs to make it work with GitLab registry or does it need any packages changes to gitlab app or docker registry app ?

                                                                        mario 1 Reply Last reply Reply Quote 2
                                                                        • mario
                                                                          mario App Dev @girish last edited by

                                                                          @girish doesn't seem like it'll need package changes, documentation will be enough.

                                                                          mario 1 Reply Last reply Reply Quote 0
                                                                          • mario
                                                                            mario App Dev @mario last edited by mario

                                                                            I take that back, I did add some package changes. Had no time to test, but things seem to be working ok from the initial glimpse at it:

                                                                            https://git.cloudron.io/cloudron/docker-registry-app/-/merge_requests/1

                                                                            Please test and report back @girish and others 🙂

                                                                            1 Reply Last reply Reply Quote 2
                                                                            • robi
                                                                              robi last edited by

                                                                              Might be useful to add auto deletion of old images:
                                                                              https://github.com/jeffstephens/retention-manager

                                                                              Life of Advanced Technology

                                                                              mario 1 Reply Last reply Reply Quote 0
                                                                              • mario
                                                                                mario App Dev @robi last edited by

                                                                                @robi GitLab does that for me 😛 Maybe a separate app? 🙂

                                                                                girish robi 2 Replies Last reply Reply Quote 0
                                                                                • girish
                                                                                  girish Staff @mario last edited by

                                                                                  @mario Just looking into this now.

                                                                                  Wondering, what is the best way forward. The app has no UI, but can have a login screen (via proxyAuth). So, when they login, they see a blank screen. Not ideal. Does it make sense to bundle any of the docker uis like https://github.com/Joxit/docker-registry-ui/ ? Seems quite easy to do. I can look into it.

                                                                                  mario 1 Reply Last reply Reply Quote 2
                                                                                  • robi
                                                                                    robi @mario last edited by

                                                                                    @mario that's great, but the standalone private registry app that's coming may need it and as @girish pointed out a simple UI.

                                                                                    Also not everyone is interested in the ruby laden GitLab and all it's complexity. 🙂

                                                                                    Life of Advanced Technology

                                                                                    1 Reply Last reply Reply Quote 0
                                                                                    • mario
                                                                                      mario App Dev @girish last edited by

                                                                                      @girish depends on what the community needs. I'm more than happy to have a separate registry + other things as separate apps for those who need it.

                                                                                      If I needed to pick the best registry solution with UI and everything else that's well maintained and suitable for Cloudron, I'd probably look at Quay which supports LDAP auth.

                                                                                      https://github.com/quay/quay

                                                                                      1 Reply Last reply Reply Quote 1
                                                                                      • mehdi
                                                                                        mehdi App Dev last edited by

                                                                                        I am 100% in favor of bundling a simple UI together with the registry. Even if one does not need it and wants to use the gitlab UI, there's basically nothing to lose besides a few kB of storage ^^

                                                                                        1 Reply Last reply Reply Quote 1
                                                                                        • jimcavoli
                                                                                          jimcavoli App Dev last edited by

                                                                                          Yeah, Quay and Harbor are definitely the big players in this space. Very similar products - harbor is CNCF graduated and Quay is upstream for the corresponding Red Hat product. Either (or both) would be good UI adds.

                                                                                          mario 1 Reply Last reply Reply Quote 1
                                                                                          • girish
                                                                                            girish Staff last edited by

                                                                                            Last I checked harbor was impractical to package (as in way too much effort, it's really geared for the k8s crowd). Quay is a good option, but let me get this basic docker registry out first, I am almost there.

                                                                                            1 Reply Last reply Reply Quote 3
                                                                                            • mario
                                                                                              mario App Dev @jimcavoli last edited by

                                                                                              @jimcavoli Quay afaik implements the protocol as well, so no need for registry separately.

                                                                                              1 Reply Last reply Reply Quote 0
                                                                                              • girish
                                                                                                girish Staff last edited by

                                                                                                So strange, I am getting a "invalid checksum digest format" whenever I push now to this registry. Has anyone seen such an error before?

                                                                                                The push refers to repository [xxx.xxx.xxx/cloudron/base]
                                                                                                fcdfeda3e242: Layer already exists 
                                                                                                0ea3bde29271: Layer already exists 
                                                                                                d75ccb14b8b6: Layer already exists 
                                                                                                74b4389a43ab: Layer already exists 
                                                                                                5f38ae1e1a63: Layer already exists 
                                                                                                3479c151673d: Layer already exists 
                                                                                                7a307b866f25: Layer already exists 
                                                                                                ce3a66c20e17: Layer already exists 
                                                                                                7197b970ebb9: Layer already exists 
                                                                                                16542a8fc3be: Layer already exists 
                                                                                                6597da2e2e52: Layer already exists 
                                                                                                977183d4e999: Layer already exists 
                                                                                                c8be1b8f4d60: Layer already exists 
                                                                                                invalid checksum digest format
                                                                                                
                                                                                                mario 1 Reply Last reply Reply Quote 0
                                                                                                • mario
                                                                                                  mario App Dev @girish last edited by

                                                                                                  @girish local filesystem?

                                                                                                  girish 1 Reply Last reply Reply Quote 0
                                                                                                  • girish
                                                                                                    girish Staff @mario last edited by girish

                                                                                                    @mario Yes, with the local storage. I wonder if it's something to do with the proxy auth. I am trying it without auth now.

                                                                                                    edit: indeed, something to do with the proxy auth. It works fine without proxy auth. Debugging.

                                                                                                    mehdi 1 Reply Last reply Reply Quote 0
                                                                                                    • mehdi
                                                                                                      mehdi App Dev @girish last edited by

                                                                                                      @girish Are you on 6.1 ? Maybe your 2FA implementation broke something with the basic auth ?

                                                                                                      girish 1 Reply Last reply Reply Quote 0
                                                                                                      • girish
                                                                                                        girish Staff @mehdi last edited by

                                                                                                        @mehdi yeah, i had that in mind and tried with 6.0 as well. fails the same. I am pretty sure this worked when I tested it back then, so I must have broke something !

                                                                                                        mehdi 1 Reply Last reply Reply Quote 0
                                                                                                        • mehdi
                                                                                                          mehdi App Dev @girish last edited by

                                                                                                          @girish You can try with an app-password, or try another Basic Auth ProxyAuth app, like Transmission (with an android app or a browser extension)

                                                                                                          1 Reply Last reply Reply Quote 0
                                                                                                          • girish
                                                                                                            girish Staff last edited by

                                                                                                            What I am seeing is that docker doesn't send any authorization header at all. The issue is very similar to https://stackoverflow.com/questions/55516317/docker-login-not-passing-basic-authentication-headers-to-nginx . I can curl just fine.

                                                                                                            1 Reply Last reply Reply Quote 0
                                                                                                            • girish
                                                                                                              girish Staff last edited by

                                                                                                              It seems that v2 registry auth does not use the basic bearer based authentication at all. https://docs.docker.com/registry/recipes/nginx/ is possibly obsolete, but I am trying to setup a registry from scratch now to double check.

                                                                                                              mario mehdi 2 Replies Last reply Reply Quote 2
                                                                                                              • mario
                                                                                                                mario App Dev @girish last edited by

                                                                                                                @girish it definitely can, that's how GitLab etc integration works.

                                                                                                                girish 1 Reply Last reply Reply Quote 0
                                                                                                                • mehdi
                                                                                                                  mehdi App Dev @girish last edited by

                                                                                                                  @girish Their doc indeed appears to be outdated. Different pages seem to indicate different things ...

                                                                                                                  1 Reply Last reply Reply Quote 0
                                                                                                                  • girish
                                                                                                                    girish Staff @mario last edited by girish

                                                                                                                    @mario thanks! i needed such a confident statement to help me keep looking further 🙂

                                                                                                                    I managed to get it to work. The issue is that proxyAuth on an auth fail redirects to the login page. But the docker registry wants it to return a 401 with a www-authenticate header. The header also causes issues with browsers since it starts popping up the login dialog.

                                                                                                                    In essence, even though the basic auth works, proxyAuth is not compatible. I thought about adding an flag to the manifest to have a different behavior but then again I don't like the current approach where we just install this registry and land on an empty page (any page even some static html with instructions would be better).

                                                                                                                    I ended up packaging it together the docker registry UI and a small LDAP server (from https://git.cloudron.io/cloudron/cloudron-serve). I haven't pushed the changes since they are not working entirely. But it's what I am working on in parallel with getting 6.1 out.

                                                                                                                    fbartels mehdi 2 Replies Last reply Reply Quote 4
                                                                                                                    • fbartels
                                                                                                                      fbartels App Dev @girish last edited by

                                                                                                                      @girish said in Docker registry:

                                                                                                                      I ended up packaging it together the docker registry UI and a small LDAP server

                                                                                                                      That sounds intriguing. What role does the ldap server serve? Just for auth against the registry ui?

                                                                                                                      girish 1 Reply Last reply Reply Quote 0
                                                                                                                      • mehdi
                                                                                                                        mehdi App Dev @girish last edited by

                                                                                                                        @girish I think the best approach would be to do a bit of user-agent parsing magic... Yeah, it would be quite specific for this use-case, but 🤷

                                                                                                                        girish 1 Reply Last reply Reply Quote 0
                                                                                                                        • girish
                                                                                                                          girish Staff @fbartels last edited by

                                                                                                                          @fbartels said in Docker registry:

                                                                                                                          That sounds intriguing. What role does the ldap server serve? Just for auth against the registry ui?

                                                                                                                          Yes, pretty much. It's just a proxy that redirects to login page and auths against LDAP. The code itself is very small, just ~100 lines or so.

                                                                                                                          1 Reply Last reply Reply Quote 0
                                                                                                                          • girish
                                                                                                                            girish Staff @mehdi last edited by

                                                                                                                            @mehdi Right, I considered UA string hack but I think dropping users in a blank page is a bit rough. So, my first step was to do the UA testing with nginx in the app itself. But, that brought the dreaded browser auth modal dialog which I really dislike. It's the main reason I ended up making proxyAuth in the first place 😉 So.. I ended up making a node server.

                                                                                                                            mehdi 1 Reply Last reply Reply Quote 0
                                                                                                                            • First post
                                                                                                                              Last post
                                                                                                                            Powered by NodeBB