Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    VPN tunnel for apps

    Feature Requests
    openvpn networking
    7
    135
    5837
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Lonkle
      Lonkle last edited by

      Alright, I got the container both running and entered into the database (I'm pretty sure). So Cloudron can "see" the app. I copied the exposed and host ports to the container returned at the end of createContainer while nulling them during the containers initialization.

      That seems to be enough for app discovery, now to see if I can get the reverse proxy working which will be the last piece of the puzzle I hope.

      1 Reply Last reply Reply Quote 1
      • marcusquinn
        marcusquinn last edited by

        Dig, Forest, Dig! šŸ•³ šŸ˜‚

        We're not here for a long time - but we are here for a good time :)
        Jersey/UK
        Work & Ecommerce Advice: https://brandlight.org
        Personal & Software Tips: https://marcusquinn.com

        Lonkle 1 Reply Last reply Reply Quote 1
        • Lonkle
          Lonkle @marcusquinn last edited by

          @marcusquinn I’m not positive but I think the Nguni reverse proxy is being fed a null or 0 hostPort. The exposedPort is part of Cloudron’s environment variables. Maybe I’ll add the hostPort to that and read it back during reverse proxy configuration. šŸ¤”

          1 Reply Last reply Reply Quote 0
          • Lonkle
            Lonkle last edited by

            I hope nobody finds me logging my development on this app to be annoying. I needed a place to keep all of these new concepts in my head. I’ve never worked with Docker or Cloudron before this month and I’m trying to jot down my experiences for me to read later until I complete this.

            If I hardcode the health checks to all pass. My app connected to the VPN stays connected and it’s terminal working but never gets past the ā€œStartingā€ phase until it times out in like ten minutes (the reverse proxy doesn’t work so neither does Let’s Encrypt...which then times out the app). This is because it’s reverse proxy isn’t set up correctly. I believe that the NGINX reverse proxy either gets the wrong hostPort or the wrong exposedPort or both.

            I’m actually amazed that I got this far. Because both the hostPort does make it into the database now for it to be discoverable by Cloudron. But why the reverse proxy WriteNginxReverseConfig receives different info than `apps.js’ ā€œadd-to-dbā€ function (since that happens first) is really confusing. I’ll keep working on it though. I can’t hardcode (wellll...I technically can for the same container but don’t want to).

            Another problem is that my NetworkMode to container changes only (for now) take effect upon installation. Have to figure out why that is - but I also have some guesses.

            Nobody told me box code was this hard. šŸ˜‚

            1 Reply Last reply Reply Quote 1
            • Lonkle
              Lonkle last edited by

              And I’m still using the app’s FQDN variable to hardcode its attachment to the VPN rather than its manifest ID since I can’t find the function to read an app’s manifest to get its ID. šŸ˜… So much more work to be done (I need the VPN client’s containerCreate option exposedPort(s) to be variable depending on its ID and the app connecting to it) - which is another reason I need to learn to use manifest data. But that’s a simple function I’m sure I’m missing somewhere. @girish - do you know it off the top of your head?

              1 Reply Last reply Reply Quote 0
              • mehdi
                mehdi App Dev last edited by

                I suddently have a doubt about what your goal is ...

                Are you trying to make the app itself, when it tries to connect to an external service, use the VPN, but still be exposed through normal internet?

                Or are you trying to restrict the users from accessing the app itself if they are not going through the VPN?

                1 Reply Last reply Reply Quote 0
                • Lonkle
                  Lonkle last edited by Lonkle

                  I’m trying to accomplish your first idea.

                  I made a Cloudron app called ā€œOpenVPN Clientā€ and it works perfectly. It can connect to any OpenVPN Server. I put a basic Apache configuration in front of it just to pass the Cloudron’s Health Check.

                  Now, my use case is to have a singular other app (any app, but I’m using the basic LAMP from the Cloudron store for testing) route all of that app’s traffic through my OpenVPN Client app. I understand other users may want to connect more than one other app and it’s technically possible to do so via setting their NetworkMode to container:open-vpn-id - but that introduces an extra complexity of exposing more than one port which Cloudron cannot do rn (but is technically possible if girish wanted to make those edits). I don’t mind though since my use case is a 1:1 OpenVPN Client + Other Cloudron app connection.

                  That’s the entire thing in a nutshell. I’ve had to dig into a lot of box code to understand how to make it happen but my OpenVPN Client app is completely working and even passes all the Cloudron health checks but the singular app connecting to it and routing all of its web traffic through it does not pass the Cloudron Healthcheck because a container connecting to the network of another container is forced to inherit all the network properties of that container. That means, the reverse proxy sees no hostPort (Cloudron is looking for it in the wrong place) when trying to configure the app and thus literally stays in the **Starting up...ā€ statuses (with it’s terminal working and traffic is verified routing through the OpenVPN Client).

                  robi 1 Reply Last reply Reply Quote 0
                  • robi
                    robi @Lonkle last edited by

                    @Lonk sounds like you need an internal network interface for internal liveness checks and external interface for the outgoing VPN traffic and it's checks.

                    otherwise you're forced to check liveness on the wrong app just because it's routing through it.

                    Life of Advanced Technology

                    Lonkle 1 Reply Last reply Reply Quote 0
                    • Lonkle
                      Lonkle @robi last edited by Lonkle

                      @robi Well, the thing is. The second app should pass all the health checks just fine. It’s a web app after all that is exposed to the web (right now testing with the LAMP stack’s homepage). The only reason it doesn’t pass the check (which makes Cloudron stop the container) is because Cloudron is setting up the reverse nginx config incorrectly due to it believing the Cloudron app just installed has no hostPort even though it really does.

                      The nginx reverse config needs to account for this to make the second app connected to the VPN Client app to even work. Because right now all I can do is use its terminal to prove its working. But the Cloudron network is setup in such a way that it’s needed to reverse proxy it’s hostPort to its exposedPort.

                      My newest idea is to write the Docker randomized hostPort as part of the Cloudron environment variables it holds and then use that in the Nginx Reverse Proxy config if the starting app doesn’t appear to have one.

                      Working on box code is harder than app code (especially because I can’t access its database outside of the command line and, well...I don’t really know SQL syntax). But shoutout to @nebulon for making it easier for me!

                      1 Reply Last reply Reply Quote 1
                      • Lonkle
                        Lonkle last edited by

                        This also leads to the discussion of exposedPorts in the OpenVPN Client app's side of things. Right now, it has to mirror each of it's connected containers (their can be multiple) exposed ports, but Cloudron (for good reason) has restricted app's exposed ports to one.

                        Dynamically assigning just the OpenVPN's exposed port(s) on demand (to logically sync up with the app that's connecting to it) is needed here and does 100% require a restart of the OpenVPN Client app unless we decide to go the route of opening x amount of the common ports and restricting multiple Cloudron app's from connecting to the OpenVPN Client to one exposed port per app (which ironically could be worked around with a dynamically assigned port forwarding to the real second containers port which then finally reverse proxies onto the hostPort + port 80 / 443 - but that seems like two much work just to connect more than one app per exposedPort at a time).

                        1 Reply Last reply Reply Quote 0
                        • mehdi
                          mehdi App Dev last edited by

                          (Disclaimer: I am no expert on docker networking magic)

                          I think instead of trying to find a way to expose the ports from the OpenVPN container, you should instead try and find a way to make it work directly from the app container itself.

                          Even if you do make it work for the exposedPort (which, I think, refers to the main web port exposed by the app, the one which is behind the reverse proxy, to expose its web interface and such, that's why there's only one: there's only one web interface), you're gonna have trouble with the extra ports (the ones defined in the manifest here https://docs.cloudron.io/custom-apps/manifest/#tcpports ) for apps that use these.

                          Lonkle mehdi 2 Replies Last reply Reply Quote 0
                          • Lonkle
                            Lonkle @mehdi last edited by Lonkle

                            @mehdi Well, that would only need to be a box discussion if we wanted more than one other Cloudron app to connect to it at a time. It'd be an interesting discussion on how to make it work with no caveats. But I personally don't see a need to have more than one container connect to the OpenVPN Client container. It's not like you can't add a secondary VPN Client Cloudron app and connect it to another app if you really need more than one VPN connection, ya know?

                            Since that's how I'll be coding it, as soon as someone tells me how to use the app object in box to "get" the cloudron manifest ID. šŸ˜‚ I'm assuming only girish or nebulon know that off the top of their head though so I'll have to reverse engineer that as soon as I figure out how to get the app passed this reverse nginx configuration barrier (which from the code looks like it's reading hostPort as null which is why no apps are working). It would be nice in that situation to know the function to get the hostPort of another container based on manifest id (my OpenVPN Client app to be exact). But once I RE those last two functions, I can do this.

                            1 Reply Last reply Reply Quote 0
                            • Lonkle
                              Lonkle last edited by Lonkle

                              Maybe these functions exist, maybe they don't. REing is hard. šŸ˜‚ I've never really coded in Node (or used Docker) before this month so these are all new concepts to me.

                              1 Reply Last reply Reply Quote 0
                              • Lonkle
                                Lonkle last edited by Lonkle

                                Oh, I think the answer to my first question was really simple: app.manifest.id. šŸ¤¦ā€ā™€ļø

                                Last thing is - I still need to find a way to get either container environment variables or the app object via only knowing the the cloudron manifest id. Then this should work perfectly.

                                1 Reply Last reply Reply Quote 0
                                • Lonkle
                                  Lonkle last edited by

                                  I mean, actually that would create a possible more than one object returned scenario.

                                  That's fine though, I'll just take the first app of that mainfest id (since more than one Cloudron app with the same manifest id can be installed simultaneously).

                                  1 Reply Last reply Reply Quote 0
                                  • Lonkle
                                    Lonkle last edited by

                                    My bad, nginx is writting the httpPort fine and it's configuration looks perfect. I'm missing something related to nginx though because all my HTTP requests to the Cloudron app that is supposed to be routed through the OpenVPN Client app just ends in a "502 Bad Gateway".

                                    1 Reply Last reply Reply Quote 0
                                    • Lonkle
                                      Lonkle last edited by Lonkle

                                      Other than this mysterious nginx issue (that I think has to do with the exposed port not being available somewhere box assumes it's supposed to be), the app will continue to work and it's terminal will run. So this is all working as it should, except for something about nginx.

                                      You know the crazy part, it's even getting an SSL cert (it must be using some DNS validation for that since the proxy isn't working).

                                      1 Reply Last reply Reply Quote 1
                                      • P
                                        p44 translator @Lonkle last edited by

                                        @Lonk said in VPN tunnel for apps:

                                        @p44 I built the OpenVPN Client you can use to change your IP for your app. @girish offered to add a custom feature to Cloudron specifically to allow Cloudron apps to network connect to my OpenVPN Client app. So, not much longer now! I might try to dive into the network code myself but I'll admit, Cloudron's base code intimidates me. šŸ˜…

                                        @Lonk Thank's a lot this is a very good feature to use. I'm following this thread. I'll wait for first release with a good web interface šŸ™‚

                                        Lonkle 1 Reply Last reply Reply Quote 0
                                        • Lonkle
                                          Lonkle @p44 last edited by Lonkle

                                          @p44 It won't be put on the Cloudron Store without a web interface for you to change VPN credentials. That's my 1.0 "this is a finished app" final goal.

                                          Right now, I'm just trying to make Cloudron's reverse proxy for the "app-connected-to-the-vpn-client" work correctly.

                                          So, as it stands, reverse proxy is forwarding requests to port 80, 443 to app.httpPort (a randomized container port, for example 34222), and then what's supposed to happen is port "34222" is supposed to proxy to the real exposedPort but that last step is what it's missing so I just get a "502 Bad Gateway". I can't find any references in the code to how to make the reverse proxy take that exposedPort from another app (the OpenVPN Client app) and use it to continue the proxy-ing. In fact, I can't even RE how official apps are doing so.

                                          This may be secret knowledge that only @girish knows and he'll have to tell me so I can fix this final proxy issue.

                                          It's funny because I'm doing so much work in box right now to get this working. But I'm mainly just REin box code. The changes required to make the OpenVPN client work / function (which it already does...just without a web interface because I can't figure out how to proxy to it's exposedPort) - like, literally the changes to box amount to 5 new lines of code. I'm guessing fixing this last "proxy-to-exposed-port" issue will be a single line of code fix making the total 6 lines of code. So the changes are so simple, it's the knowledge of box networking I'm lacking in. Cloudron uses an NPM library called dockerode to do it's Docker stuff. I thought it was the problem, but it's not - it's working just fine. It's this silly proxy issue. I'm so close...yet so far.

                                          1 Reply Last reply Reply Quote 0
                                          • Lonkle
                                            Lonkle last edited by Lonkle

                                            Talked to @girish a little bit to confirm that the ExposedPort binds to the app.httpPort via dockerode.createContainer magic. Soooo...there's only one solution. Use dockerode's own library to fix this problem. How to do so...I'm not sure, time to look into dockerode's library. But, the .run command looks promising.

                                            1 Reply Last reply Reply Quote 0
                                            • Lonkle
                                              Lonkle last edited by

                                              I can't believe this whole time it wasn't a Reverse Nginx Issue. šŸ˜‚ I had a sneaking suspicion it was a dockerode issue but I didn't want it to be. I'm scared of making any dockerode changes.

                                              1 Reply Last reply Reply Quote 0
                                              • Lonkle
                                                Lonkle last edited by

                                                This is likely what I need to use (conditionally when connecting the OpenVPN Client app to another Cloudron app):

                                                https://github.com/apocas/dockerode/blob/409a08e310645f5a82234bc3a49b40c98fb98cdd/lib/docker.js#L1432

                                                Closer than I've ever been now. But I still don't know how to use that command yet.

                                                1 Reply Last reply Reply Quote 0
                                                • Lonkle
                                                  Lonkle last edited by Lonkle

                                                  Or...or, what if I forced the ExposedPort to mirror the httpPort. That would solve the Nginx "502 Bad Gateway" issue (which is the last hurdle for a perfectly working PoC). I mean, is there really any reason for an exposed port in a Cloudron sense since it just reverse proxies all apps exposed ports to 80, 443 anyway. I think I'm going to try this first instead of the whole diving into dockerode thing.

                                                  1 Reply Last reply Reply Quote 0
                                                  • Lonkle
                                                    Lonkle last edited by Lonkle

                                                    Okay...there is a tiny reason. More like a medium reason. More like a BIG reason. Which is, the web apps themselves most-of-the-time choose to listen only on their exposed port (which means, I'd still need to manually proxy their exposed port to their app.httpPort...somehow). But I'll deal with that later. Gotta try this "mirror ports" experiment first to even see if it works.

                                                    Making a PoC is more important than those logistics rn tho.

                                                    robi 1 Reply Last reply Reply Quote 0
                                                    • robi
                                                      robi @Lonkle last edited by

                                                      @Lonk what if you virtualized the exposed port? another nginx that you control?

                                                      Life of Advanced Technology

                                                      Lonkle 1 Reply Last reply Reply Quote 1
                                                      • Lonkle
                                                        Lonkle @robi last edited by Lonkle

                                                        @robi That's a great idea. I've thought about it before, and I'm 99% sure it would work...I just can't wrap my head around it (it's roughly what I meant in my last post when I said " I'd still need to manually proxy their exposed port to their app.httpPort...somehow").

                                                        Tbh, I think I burnt myself out because I know you're right. I mean, putting another nginx reverse proxy behind the randomized app.httpPort (say 32344) to proxy_pass to an exposed port (say 8080) would so work. My brain is just having trouble processing the concept rn (as in, where I'd need to put that code since I think it would still have to be created at the box level).

                                                        1 Reply Last reply Reply Quote 0
                                                        • Lonkle
                                                          Lonkle last edited by

                                                          So, your idea is my next attempt at making the OpenVPN client universal.

                                                          Oh, and I need to find a way to make Cloudron expose all of OpenVPN Client's internal ports. Or at least the common ones - because the second app inherits the exposed ports of OpenVPN Client app (so the two apps would need to match up their exposed ports somehow, and a wildcard, open all ports on the OpenVPN Client's side seems like the only way to do so).

                                                          1 Reply Last reply Reply Quote 0
                                                          • Lonkle
                                                            Lonkle last edited by

                                                            I feel like I live-blogged my entire RE and development process with this thread. šŸ˜‚

                                                            1 Reply Last reply Reply Quote 1
                                                            • robi
                                                              robi last edited by

                                                              @Lonk said in VPN tunnel for apps:

                                                              @robi That's a great idea. I've thought about it before, and I'm 99% sure it would work...I just can't wrap my head around it.

                                                              Take a break. Please šŸ™‚ A nap, walk, anything.

                                                              @Lonk said in VPN tunnel for apps:

                                                              So, your idea is my next attempt at making the OpenVPN client universal.

                                                              Right on!

                                                              Oh, and I need to find a way to make Cloudron expose all of OpenVPN Client's internal ports.
                                                              That means Nginx goes into the OpenVPN container, no?
                                                              Rules for reverse-proxying are something along the lines of:
                                                              if coming from internal IPs, do X
                                                              if coming from cloudron IP, do Y
                                                              if coming from elsewhere, do Z to the App.

                                                              Life of Advanced Technology

                                                              Lonkle 2 Replies Last reply Reply Quote 0
                                                              • Lonkle
                                                                Lonkle @robi last edited by

                                                                @robi said in VPN tunnel for apps:

                                                                That means Nginx goes into the OpenVPN container, no?
                                                                Rules for reverse-proxying are something along the lines of:
                                                                if coming from internal IPs, do X
                                                                if coming from cloudron IP, do Y
                                                                if coming from elsewhere, do Z to the App.

                                                                I'm new to nginx, literally only using it's .conf files to try to get this working as a mere proof of concept. But so far I'm only familiar with proxy_pass for it taking incoming 80, 443 ports to pass it to the internal Docker port (and IP...I think but I don't *think this is an IP issue, though it might be).

                                                                1 Reply Last reply Reply Quote 0
                                                                • Lonkle
                                                                  Lonkle @robi last edited by Lonkle

                                                                  @robi But if I wanted to expose all of it's ports, it would actually be more of a box docker.js thing with the "exposed ports" parameter being fed to gContainer.createContainer. I should be able to do that after I give my mind a lil break.

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • Lonkle
                                                                    Lonkle last edited by Lonkle

                                                                    Okay, after hardcode exposing ALL internal ports of the OpenVPN container, now the last thing is adding a second nginx reverse proxy which would mean this:

                                                                    My exposedPort for the Gucamole app I'm installing to connect to the OpenVPN Client app exposes 8080. It gets a Docker internal httpPort of 32455. Nginx already correctly creates a nginx proxy config from web incoming port 80 and 443 to 32455 (somehow, presumably DNS TXT records, it even gets an SSL cert).

                                                                    So, I need to have nginx after that continue to proxy it to 8080. Which normally is Docker's job (with it's binding port) but when connecting two container's networks together, it doesn't do that job with the dockerode library for some Docker-y reason. I wonder...can I reverse proxy it in the same file as the original reserve proxy server. Or do I have to create a .conf and an entirely new one?

                                                                    Lonkle 1 Reply Last reply Reply Quote 0
                                                                    • Lonkle
                                                                      Lonkle @Lonkle last edited by Lonkle

                                                                      If I have the same nginx reverse proxy "listen" to it's own forwarded port (32455), then I could proxy_pass again to the real internal port and IP (8080) of Guacomole (the test app I'm now using to connect to the vpn client app)...or maybe I need another reverse proxy residing at Internal-IP-of-Guacomole:32455 to then proxy_pass again to 8080.

                                                                      1 Reply Last reply Reply Quote 0
                                                                      • robi
                                                                        robi last edited by

                                                                        The problem there is that the stuff in the container doesn't have access outside the container, so it's hard to drop updates to the host Nginx as another .conf file.

                                                                        You could tell box to do it via how it already happens for new apps. Must be an API for it.

                                                                        Life of Advanced Technology

                                                                        1 Reply Last reply Reply Quote 0
                                                                        • Lonkle
                                                                          Lonkle last edited by

                                                                          @robi I keep trying to experiment on how to shoehorn a second Ngnix proxy into this to get it all up and running 100% with no success. 😢

                                                                          Inter-communication between the two containers would be nice, but tbh I could hardcode that if I knew how to setup the second reverse nginx inside of reverseproxy.js. Which, honestly, I can't figure out how.

                                                                          1 Reply Last reply Reply Quote 0
                                                                          • robi
                                                                            robi last edited by

                                                                            maybe we need a few more eyes from @nebulon @girish @mehdi for more options šŸ¤—

                                                                            Life of Advanced Technology

                                                                            1 Reply Last reply Reply Quote 0
                                                                            • Lonkle
                                                                              Lonkle last edited by

                                                                              I know the answer lies here:

                                                                              https://git.cloudron.io/cloudron/box/-/blob/master/src/reverseproxy.js#L463

                                                                              I can configure a second file for a second nginx server to proxy app.httpPort to app.manifest.httpPort - but, like, I don't know how to change that it's hardcoded to listen only on 80 and 443. Plus, I don't know how to actually get this hypothetical second nginx reverse proxy running to "take" the new config within the container that's connecting to the VPN. But that's the last step here on how to make this work.

                                                                              1 Reply Last reply Reply Quote 0
                                                                              • mehdi
                                                                                mehdi App Dev @mehdi last edited by

                                                                                @mehdi said in VPN tunnel for apps:

                                                                                (Disclaimer: I am no expert on docker networking magic)

                                                                                I think instead of trying to find a way to expose the ports from the OpenVPN container, you should instead try and find a way to make it work directly from the app container itself.

                                                                                Even if you do make it work for the exposedPort (which, I think, refers to the main web port exposed by the app, the one which is behind the reverse proxy, to expose its web interface and such, that's why there's only one: there's only one web interface), you're gonna have trouble with the extra ports (the ones defined in the manifest here https://docs.cloudron.io/custom-apps/manifest/#tcpports ) for apps that use these.

                                                                                I really think this. You should try finding a way to make the app container's default route be through the OpenVPN network, but still be connected to cloudron's regular network so it can interact with nginx and stuff. There is no good reason to make all the traffic, even the local one, be through OpenVPN.

                                                                                @Lonk is there a repo with your box changes ? Maybe I'll have time to take a look this WE

                                                                                Lonkle 4 Replies Last reply Reply Quote 0
                                                                                • Lonkle
                                                                                  Lonkle @mehdi last edited by

                                                                                  @mehdi You only have to change one file (docker.js) so I’ll send that to you today!

                                                                                  But the app has no other issues. It is on the Cloudron network. It has everything network-wise that the OpenVPN container has which means there are no issues anywhere. The only issue is that Nginx doesn’t take that into account because it inherits all network ports and modes from the OVPN Client it’s connected to.

                                                                                  I actually have an idea though to prove that everything works except the web interface using the app terminal.

                                                                                  I will also be sending you my OVPN client in chat so you can see how it works.

                                                                                  1 Reply Last reply Reply Quote 0
                                                                                  • Lonkle
                                                                                    Lonkle @mehdi last edited by

                                                                                    @mehdi Thanks for your willingness to help me out. You were my inspiration for working on this. Felt right since you had to dive into box code to add the NET_ADMIN capability.

                                                                                    1 Reply Last reply Reply Quote 1
                                                                                    • Lonkle
                                                                                      Lonkle @mehdi last edited by Lonkle

                                                                                      @mehdi Oh, and just to make it clear, the app is running fine even in the terminal. If I install the LAMP app for instance and connect it to the OpenVPN Client. I can go to LAMP's terminal and type curl 127.0.0.1:80 and it will give me the LAMP Welcome page, but from the IP address of the OpenVPN Client. It'll just never get past the "Starting..." stage without a 200 response and Nginx is routing it to the wrong location. Maybe I could try hardcoding it to the right port after it installs it's config. šŸ¤”

                                                                                      Like, this works. But I don't know how to get NGINX to route it to the port I need it to route to. Docker is supposed to bind the app.httpPort (say 34567) to the exposedPort (say 80). But that Docker binding command has to be removed to connect the containers so NGINX has to bind them manually somehow and I don't know how to work with Nginx.

                                                                                      1 Reply Last reply Reply Quote 0
                                                                                      • Lonkle
                                                                                        Lonkle last edited by Lonkle

                                                                                        I'm near positive the way to route NGINX properly is to to add another reverse proxy which leads from the Docker randomized app.http port to the exposedPort (AKA app.manifest.httpPort). I'm not sure how though. But I can clearly access the exposedPort's data from either terminal.

                                                                                        In the terminal of either of the two network-bonded apps (the LAMP or the VPN Client) - I can curl (localhost) the exposed ports of either app (the VPN Client or the LAMP) and get back the correct HTML pages.

                                                                                        It's just that the reverse proxy is forwarding it to a port that Docker wasn't allowed to bind. So with no binding, that means no NGINX web page.

                                                                                        1 Reply Last reply Reply Quote 0
                                                                                        • Lonkle
                                                                                          Lonkle last edited by Lonkle

                                                                                          For people just joining and for me to read this in the future:

                                                                                          Cloudron has a user-defined network cloudron that it uses for all of it's apps and services. I'm connecting my OpenVPN Client app directly to another app (LAMP for testing) using NetworkMode which is the official way to do this. When doing so, both apps share the exact same network space (including both being connected to the cloudron network) and can, in fact, even talk to each other directly (I need to set the OpenVPN Client's exposed port to something ridiculous like in the 50000s so it doesn't conflict with any regular app's commonly exposed ports - but in the end the OpenVPN Client will expose all of it's internal network ports -P for compatibility with all other apps connecting to it - but for now, this will do since I'm just testing).

                                                                                          Now, the NGINX Reverse Proxy resides on the main box level of Cloudron and routes to the randomized Docker binding port of the app which forwards it to it's "real" port (I honestly don't know where the IP translation takes place...I can't find it). šŸ˜…

                                                                                          So, solution, place a second NGINX proxy at the randomized Docker app.http port and have that nginx server be inside of the internal network of the LAMP testing app so it can then internally forward the web request to it's exposedPort which is, also, port 80 (for LAMP anyway).

                                                                                          1 Reply Last reply Reply Quote 0
                                                                                          • Lonkle
                                                                                            Lonkle last edited by

                                                                                            ^^^^ -- Does anyone know how to do this?

                                                                                            1 Reply Last reply Reply Quote 0
                                                                                            • Lonkle
                                                                                              Lonkle @mehdi last edited by Lonkle

                                                                                              @mehdi said in VPN tunnel for apps:

                                                                                              There is no good reason to make all the traffic, even the local one, be through OpenVPN.

                                                                                              All local traffic is local, it doesn't go through the VPN. All of the connected-to-the-vpn-client apps run indefinitely. Eventually after Cloudron says "Starting..." it switches the message to "Not Responding" but the app will continue indefinitely because there's nothing actually wrong with it. It's connecting to all the local services it needs, it's just, this feature was designed without consideration of Cloudron's network. That's why from within the app's terminal, I can access everything locally and everything externally (the open web) from Cloudron and the web sees my VPN Client's IP Address instead of Cloudrons. This is a perfectly running app. Except...I can't expose it's internal IP:ExposedPort to NGINX to have it actually pass the health check (meaning the web app isn't being routed to the right place and I think I need a second NGINX reverse proxy to get me routed to the app's exposed port). It gets as far as app.httpPort and then because there is no binding for the app.mainfest.httpPort (AKA: ExposedPort), it just gets stuck on the web side of things.

                                                                                              I can curl all I want into it the LAMP welcome. But can't access it from outside the container itself.

                                                                                              mehdi 1 Reply Last reply Reply Quote 0
                                                                                              • Lonkle
                                                                                                Lonkle last edited by Lonkle

                                                                                                Quick example of all local processes running for the second container:

                                                                                                Oct 02 12:05:28 2020-10-02 16:05:28,718 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
                                                                                                Oct 02 12:05:28 2020-10-02 16:05:28,719 INFO success: redis-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
                                                                                                Oct 02 12:05:40 [GET] /healthcheck

                                                                                                It just gets stuck at the health check because NGINX is giving it a 502 Bad Gateway since port 80 is not bound to port app.httpPort. That's what I still need to figure out and I think what's best is @robi's double nginx server suggestion. I just don't know how to set up NGINX inside of the LAMP container from box code though I'm sure it's possible.

                                                                                                1 Reply Last reply Reply Quote 0
                                                                                                • Lonkle
                                                                                                  Lonkle last edited by Lonkle

                                                                                                  I still think the solution lies in this function:

                                                                                                  https://git.cloudron.io/cloudron/box/-/blob/master/src/reverseproxy.js#L463

                                                                                                  But that function is setting NGINX config listening on the server's public IP port. I need NGINX inside of the container to forward to the app.manifest.httpPort (in LAMP's case it's port 80).

                                                                                                  Or maybe it doesn't need to be inside of the container...since if I SSH into the VPS and do curl ip-address-of-lamp:80 then it returns the HTML just fine.

                                                                                                  1 Reply Last reply Reply Quote 0
                                                                                                  • Lonkle
                                                                                                    Lonkle last edited by

                                                                                                    SUCCESS!!!

                                                                                                    I'm so close. I can manually edit the NGINX configuration of the LAMP-test app and get it's web app working using it's local IP (not 127.0.0.1:app.httpPort, but 172.18.0.3:80 - the port 80 is what LAMP uses). Both web pages come up for their each individual domain names. WOOOO!

                                                                                                    It still doesn't make it past the "starting..." step though. That might be because I'm editing the NGINX file and then restarting the app to make it's web interface work. If the NGINX file was properly created from the beginning the check might pass.

                                                                                                    1 Reply Last reply Reply Quote 0
                                                                                                    • Lonkle
                                                                                                      Lonkle last edited by

                                                                                                      Which...now I need to find the function to get the IP address of the VPN container (the app connected to it share's it's IP address).

                                                                                                      1 Reply Last reply Reply Quote 0
                                                                                                      • mehdi
                                                                                                        mehdi App Dev @Lonkle last edited by

                                                                                                        @Lonk said in VPN tunnel for apps:

                                                                                                        @mehdi said in VPN tunnel for apps:

                                                                                                        There is no good reason to make all the traffic, even the local one, be through OpenVPN.

                                                                                                        All local traffic is local, it doesn't go through the VPN.

                                                                                                        Sorry, I meant: There is no good reason to make all the traffic, even the local one, be through the OpenVPN client container.

                                                                                                        1 Reply Last reply Reply Quote 0
                                                                                                        • mehdi
                                                                                                          mehdi App Dev last edited by

                                                                                                          What you should try to achieve is making only the app's outgoing traffic go through OpenVPN, and the connexion between it and Nginx and stuff stay exactly the same as it currently is

                                                                                                          Lonkle 1 Reply Last reply Reply Quote 0
                                                                                                          • Lonkle
                                                                                                            Lonkle @mehdi last edited by Lonkle

                                                                                                            @mehdi said in VPN tunnel for apps:

                                                                                                            What you should try to achieve is making only the app's outgoing traffic go through OpenVPN, and the connexion between it and Nginx and stuff stay exactly the same as it currently is

                                                                                                            That is currently the case as far as I can tell. All incoming traffic bypasses the VPN entirely. Outgoing traffic uses the VPN's connection. But the NGINX needs an incredibly minor change. There's no way around that.

                                                                                                            girish 1 Reply Last reply Reply Quote 0
                                                                                                            • Lonkle
                                                                                                              Lonkle last edited by Lonkle

                                                                                                              But the bigger deal is that it works now!

                                                                                                              Screen Shot 2020-10-02 at 6.18.06 PM.png

                                                                                                              Okay, I know it says not responding but it actually is fully working (web page and everything). The "not responding" status is something I think will fix itself when I get the Nginx IP proxy_pass working. Because it's sort of a race condition as to when I start the app and when I manually rewrite the Nginx IP. And I'll never go fast enough to beat the "Not responding message". šŸ˜‚

                                                                                                              1 Reply Last reply Reply Quote 1
                                                                                                              • Lonkle
                                                                                                                Lonkle last edited by Lonkle

                                                                                                                I just need a function like getAppByManifestId(manifestID) - and this will be a fully functioning proof of concept. And then another function for getLocalIPAddressByApp(app).

                                                                                                                Oh, and I need to change some slight NGINX config, make it more dynamic (Cloudron hardcodes it's proxy_pass to 127.0.0.1 rn and it needs to be dynamic).

                                                                                                                1 Reply Last reply Reply Quote 0
                                                                                                                • Lonkle
                                                                                                                  Lonkle last edited by

                                                                                                                  @girish The post above this one, do you know off the top of your head how to get those values?

                                                                                                                  1 Reply Last reply Reply Quote 0
                                                                                                                  • girish
                                                                                                                    girish Staff last edited by

                                                                                                                    You can't getAppByManifestId since there can be many apps for a given manifest id. I think you have to store whatever you want in the database in the apps table. Then when you get the app, you can get that information (like say the network id). Sorry, if my answer is totally off since I didn't read the full thread completely.

                                                                                                                    Lonkle 4 Replies Last reply Reply Quote 0
                                                                                                                    • Lonkle
                                                                                                                      Lonkle @girish last edited by Lonkle

                                                                                                                      @girish I had actually nailed that issue out actually as a matter of fact. The only thing I have left is getLocalIPAddressByApp(app) - but maybe the app object already stores its internal IP (I didn’t think it did, but it stores a lot)?

                                                                                                                      1 Reply Last reply Reply Quote 0
                                                                                                                      • Lonkle
                                                                                                                        Lonkle @girish last edited by

                                                                                                                        @girish As for the first function, it was mainly due to me needing to find a way to identify the VPN client app by the only unique identifier I knew. I’ve hardcoded it for my use case rn...but that won’t work for the production version now that I think about it - so I’ll still need to get app data using the manifest.id (com.joelstickney.openvpn-client-cloudron). So when the app that needs to connect to it starts, it attaches to the container ID of the Open VPN client on containerCreate.

                                                                                                                        Maybe right now I’ll store the things I need in ENV variables and find a way to use the ā€œappsā€ object to parse through all of them till I find the VPN client. šŸ¤”

                                                                                                                        1 Reply Last reply Reply Quote 0
                                                                                                                        • Lonkle
                                                                                                                          Lonkle @girish last edited by

                                                                                                                          @girish I think you’re right about using the app table. I’ll try to RE how you use it to populate the apps variable.

                                                                                                                          And no worries not following along with this thread. It’s been really difficult for me to get this working so I used this as kind of note-taking. Now that I’m nearly finished, people won’t get as bugged by my posting here every time I discover something new about the way box works. šŸ˜‚

                                                                                                                          1 Reply Last reply Reply Quote 0
                                                                                                                          • Lonkle
                                                                                                                            Lonkle last edited by

                                                                                                                            Oh wait, the container object, would contain the IP and the container ID. I could intercept that right after it’s created. 🧐

                                                                                                                            1 Reply Last reply Reply Quote 0
                                                                                                                            • First post
                                                                                                                              Last post
                                                                                                                            Powered by NodeBB