Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved HashiCorp Vault

    App Wishlist
    5
    32
    1146
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • mario
      mario App Dev last edited by girish

      https://github.com/hashicorp/vault

      ultraviolet 1 Reply Last reply Reply Quote 1
      • ultraviolet
        ultraviolet App Dev @mario last edited by

        @mario this would be useful and I was looking at this a week ago. Vault uses MLOCK to stop the data from being sent to disk, like using swap memory, in order to get MEMLOCK you need to tell docker to provide that privilege to the container. Which from what I can see is a limitation on Cloudron (for good reasons). I have the package done and I can share it out once I have cleaned the code up a little but in order to get Vault to run, MLOCK capabilities needs to be disabled in the config file.

        You could argue that the docker container in Cloudron is self-contained but something like this would be better hosted internally behind a firewall than something you would put up onto DO for example.

        Happy to take differing options and/or opinions.

        1 Reply Last reply Reply Quote 2
        • ultraviolet
          ultraviolet App Dev last edited by

          Actually, I went and RTFM: https://cloudron.io/documentation/storage/ I might be able to add custom properties but that would mean all the containers would have the priv.

          @girish can you shed any light on the --cap-add=IPC_LOCK being passed to docker?

          thanks

          mehdi 1 Reply Last reply Reply Quote 1
          • mehdi
            mehdi App Dev @ultraviolet last edited by

            @ultraviolet take a look at https://cloudron.io/documentation/custom-apps/manifest/#capabilities

            This mechanism is already used for some apps (at least OpenVPN, which also needs a capability for networking stuff). The MLOCK capability is currenlty not one of the possibilties, but it would be trivial to add to a future version of cloudron

            ultraviolet 1 Reply Last reply Reply Quote 1
            • ultraviolet
              ultraviolet App Dev @mehdi last edited by

              @mehdi awesome thanks.

              1 Reply Last reply Reply Quote 1
              • ultraviolet
                ultraviolet App Dev last edited by

                If anyone is interested the repo is here https://github.com/euanmcgregor/vault-cloudron

                It is working, I am just looking at LDAP integration just now.

                mario 1 Reply Last reply Reply Quote 3
                • mario
                  mario App Dev @ultraviolet last edited by

                  @ultraviolet you're a star, thank you so much.

                  Thanks for the additional clarifications on the manifest capabilities @mehdi

                  If @girish and @nebulon can add the capability for MLOCK and you can get the LDAP working, that would be beyond awesome.

                  1 Reply Last reply Reply Quote 1
                  • girish
                    girish Staff last edited by

                    @ultraviolet Thanks! I have added mlock capability for 5.4 in https://git.cloudron.io/cloudron/box/-/commit/84f41e08cfbd3eb4dd5ac2d0e949b6a6d724f1c7

                    For now (until 5.4) is out, I guess you can disable mlock using disable_mlock ? (Atleast as per https://github.com/hashicorp/docker-vault/issues/24)

                    ultraviolet 1 Reply Last reply Reply Quote 1
                    • ultraviolet
                      ultraviolet App Dev @girish last edited by

                      @girish that is awesome stuff. I have disabled mlock in the config to get to get it going, once 5.4 is out I will adjust the manifest file.

                      Just working on LDAP integration!

                      1 Reply Last reply Reply Quote 1
                      • girish
                        girish Staff last edited by

                        @ultraviolet It seems we need to make another 5.3 release because of some issues. So, maybe I can put this in 5.3 as well. Let me see.

                        ultraviolet 1 Reply Last reply Reply Quote 1
                        • ultraviolet
                          ultraviolet App Dev @girish last edited by

                          @girish awesome. I can see that the change is in 5.3 and I have just upgraded. Made a small change but it is still throwing the error, my manifest file shows this now:

                          
                          {
                            "id": "com.vault.cloudron",
                            "version": "0.1.0",
                            "minBoxVersion": "5.3.0",
                            "healthCheckPath": "/",
                            "httpPort": 8200,
                            "addons": {
                              "localstorage": {},
                              "ldap": {}
                            },
                            "capabilities": [
                              "mlock"
                            ],
                            "manifestVersion": 2
                          }
                          

                          Not sure if I am missing something here?

                          1 Reply Last reply Reply Quote 0
                          • girish
                            girish Staff last edited by

                            @ultraviolet Let me test it out and get back. How should I test this? Just build the app and install it and it will fail ?

                            1 Reply Last reply Reply Quote 0
                            • ultraviolet
                              ultraviolet App Dev last edited by

                              pretty much yeah, the services won't start and the logs will show an error message

                              fbartels 1 Reply Last reply Reply Quote 0
                              • fbartels
                                fbartels App Dev @ultraviolet last edited by fbartels

                                @ultraviolet I think I got a littler closer to a working state. Currently restarting since my Cloudron wanted a reboot after the last update.

                                https://github.com/euanmcgregor/vault-cloudron/pull/1

                                edit: hmm, no. this is what is logged:

                                Jun 29 20:54:05 2020-06-29 18:54:05,044 INFO spawned: 'vault' with pid 12
                                Jun 29 20:54:05 Error initializing core: Failed to lock memory: cannot allocate memory
                                Jun 29 20:54:05
                                Jun 29 20:54:05 This usually means that the mlock syscall is not available.
                                Jun 29 20:54:05 Vault uses mlock to prevent memory from being swapped to
                                Jun 29 20:54:05 disk. This requires root privileges as well as a machine
                                Jun 29 20:54:05 that supports mlock. Please enable mlock on your system or
                                Jun 29 20:54:05 disable Vault from using it. To disable Vault from using it,
                                Jun 29 20:54:05 set the `disable_mlock` configuration option in your configuration
                                Jun 29 20:54:05 file.
                                Jun 29 20:54:05 2020-06-29 18:54:05,115 INFO exited: vault (exit status 1; not expected)
                                Jun 29 20:54:06 2020-06-29 18:54:06,118 INFO spawned: 'vault' with pid 23
                                
                                ultraviolet 1 Reply Last reply Reply Quote 0
                                • ultraviolet
                                  ultraviolet App Dev @fbartels last edited by

                                  @fbartels if you change the variable disable_mlock in the start.sh to true it should start but the new capability will negate this.

                                  fbartels 1 Reply Last reply Reply Quote 0
                                  • fbartels
                                    fbartels App Dev @ultraviolet last edited by

                                    @ultraviolet yes, that is the workaround I am using at the moment as well.

                                    ultraviolet 1 Reply Last reply Reply Quote 0
                                    • ultraviolet
                                      ultraviolet App Dev @fbartels last edited by

                                      @fbartels cool will wait and see what @girish finds.

                                      fbartels 1 Reply Last reply Reply Quote 0
                                      • fbartels
                                        fbartels App Dev @ultraviolet last edited by

                                        @ultraviolet do you have ldap working already?

                                        You had the ldap script missing (not added with git add) so I tried my own, but even after config has completed I cannot login and only get Authentication failed: ldap operation failed: unable to retrieve user bind DN

                                        ultraviolet 1 Reply Last reply Reply Quote 0
                                        • ultraviolet
                                          ultraviolet App Dev @fbartels last edited by ultraviolet

                                          @fbartels I did try to get ldap going, the issue doing it automatically during install is you need to unseal and login to the vault before you can enable the LDAP. Which is hard when the login info is in a text file I am sure it is possible but my knowledge is a bit limited on that, plus it might not be idempotent.

                                          When I tried it manually it gave me that exact error I am still checking to see what the issue might be but I have kind of drawn a blank at the moment. Will take a fresh look later this week when I get a bit of time.

                                          fbartels 1 Reply Last reply Reply Quote 1
                                          • girish
                                            girish Staff last edited by

                                            I can reproduce it. The container definitely has the IPC_LOCK caps, maybe it needs something more. Investigating

                                            1 Reply Last reply Reply Quote 0
                                            • girish
                                              girish Staff last edited by

                                              @ultraviolet https://github.com/euanmcgregor/vault-cloudron/pull/2 fixes the mlock issue

                                              fbartels 1 Reply Last reply Reply Quote 1
                                              • fbartels
                                                fbartels App Dev @girish last edited by

                                                @girish ah, yes of course. You need to set the capability on the binary to make use of it (as non root).

                                                1 Reply Last reply Reply Quote 0
                                                • girish
                                                  girish Staff last edited by

                                                  We can probably remove the supervisor use entirely and just use gosu exec cloudron:cloudron but maybe it's there for a reason

                                                  1 Reply Last reply Reply Quote 0
                                                  • ultraviolet
                                                    ultraviolet App Dev last edited by

                                                    thanks both, I have never had a pull request before!! I am just looking at them now.

                                                    1 Reply Last reply Reply Quote 0
                                                    • fbartels
                                                      fbartels App Dev @ultraviolet last edited by fbartels

                                                      @ultraviolet I managed to get ldap login working. In the end I needed to change the lookup attribute (it weird that you can configure a search filter for groups, but not for users).

                                                      Change is in https://github.com/euanmcgregor/vault-cloudron/pull/4

                                                      Edit: OIDC login is not yet working btw.

                                                      ultraviolet 1 Reply Last reply Reply Quote 0
                                                      • ultraviolet
                                                        ultraviolet App Dev @fbartels last edited by ultraviolet

                                                        @fbartels awesome stuff on the LDAP. 👍

                                                        I have made a few more tweaks with permissions and I have merged your request too. I have also removed the initial init for the vault. It is now done via the GUI which I like better because there are no keys being added to the container plus the user experience is a bit nicer. I have still kept the logic in in case someone wants to automate it.

                                                        1 Reply Last reply Reply Quote 0
                                                        • ultraviolet
                                                          ultraviolet App Dev last edited by

                                                          Sorted, I think I have been running it for most of the day testing some stuff. Not sure if this is app store worthy but it interesting to learn about.
                                                          Thanks for your help @fbartels and @girish

                                                          1 Reply Last reply Reply Quote 0
                                                          • girish
                                                            girish Staff last edited by

                                                            @ultraviolet I think it will be a great addition to the store. Are you able to add a license file to the package? Like https://git.cloudron.io/cloudron/pixelfed-app/-/blob/master/LICENSE (MIT). You can change copyright to be yours.

                                                            Once you do that, I can fix it up and get it published.

                                                            @fbartels @ultraviolet How does the LDAP login work ? I don't see ldap-config.sh called from anywhere.

                                                            ultraviolet 1 Reply Last reply Reply Quote 0
                                                            • ultraviolet
                                                              ultraviolet App Dev @girish last edited by

                                                              @girish sweet. I will do that no problem. The LDAP has to be done manually as the vault needs to be init'd and unsealed so you would need to execute the script manually.

                                                              1 Reply Last reply Reply Quote 0
                                                              • girish
                                                                girish Staff last edited by

                                                                I will add the LDAP notes to the docs then.

                                                                ultraviolet 1 Reply Last reply Reply Quote 0
                                                                • ultraviolet
                                                                  ultraviolet App Dev @girish last edited by

                                                                  @girish License has been added to the repo.

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • girish
                                                                    girish Staff last edited by

                                                                    I have now published this as unstable! Thanks @ultraviolet . The repo is at https://git.cloudron.io/cloudron/vault-app and you should have push access already. I am writing tests before marking it as stable.

                                                                    1 Reply Last reply Reply Quote 3
                                                                    • First post
                                                                      Last post
                                                                    Powered by NodeBB