VPN tunnel for apps

[Lonkle]

My bad, nginx is writting the httpPort fine and it's configuration looks perfect. I'm missing something related to nginx though because all my HTTP requests to the Cloudron app that is supposed to be routed through the OpenVPN Client app just ends in a "502 Bad Gateway".

[Lonkle]

Other than this mysterious nginx issue (that I think has to do with the exposed port not being available somewhere box assumes it's supposed to be), the app will continue to work and it's terminal will run. So this is all working as it should, except for something about nginx.

You know the crazy part, it's even getting an SSL cert (it must be using some DNS validation for that since the proxy isn't working).

[p44]

@Lonk said in VPN tunnel for apps:

@p44 I built the OpenVPN Client you can use to change your IP for your app. @girish offered to add a custom feature to Cloudron specifically to allow Cloudron apps to network connect to my OpenVPN Client app. So, not much longer now! I might try to dive into the network code myself but I'll admit, Cloudron's base code intimidates me.

@Lonk Thank's a lot this is a very good feature to use. I'm following this thread. I'll wait for first release with a good web interface

[Lonkle]

@p44 It won't be put on the Cloudron Store without a web interface for you to change VPN credentials. That's my 1.0 "this is a finished app" final goal.

Right now, I'm just trying to make Cloudron's reverse proxy for the "app-connected-to-the-vpn-client" work correctly.

So, as it stands, reverse proxy is forwarding requests to port 80, 443 to app.httpPort (a randomized container port, for example 34222), and then what's supposed to happen is port "34222" is supposed to proxy to the real exposedPort but that last step is what it's missing so I just get a "502 Bad Gateway". I can't find any references in the code to how to make the reverse proxy take that exposedPort from another app (the OpenVPN Client app) and use it to continue the proxy-ing. In fact, I can't even RE how official apps are doing so.

This may be secret knowledge that only @girish knows and he'll have to tell me so I can fix this final proxy issue.

It's funny because I'm doing so much work in box right now to get this working. But I'm mainly just REin box code. The changes required to make the OpenVPN client work / function (which it already does...just without a web interface because I can't figure out how to proxy to it's exposedPort) - like, literally the changes to box amount to 5 new lines of code. I'm guessing fixing this last "proxy-to-exposed-port" issue will be a single line of code fix making the total 6 lines of code. So the changes are so simple, it's the knowledge of box networking I'm lacking in. Cloudron uses an NPM library called dockerode to do it's Docker stuff. I thought it was the problem, but it's not - it's working just fine. It's this silly proxy issue. I'm so close...yet so far.

[Lonkle]

Talked to @girish a little bit to confirm that the ExposedPort binds to the app.httpPort via dockerode.createContainer magic. Soooo...there's only one solution. Use dockerode's own library to fix this problem. How to do so...I'm not sure, time to look into dockerode's library. But, the .run command looks promising.

[Lonkle]

I can't believe this whole time it wasn't a Reverse Nginx Issue. I had a sneaking suspicion it was a dockerode issue but I didn't want it to be. I'm scared of making any dockerode changes.

[Lonkle]

This is likely what I need to use (conditionally when connecting the OpenVPN Client app to another Cloudron app):

https://github.com/apocas/dockerode/blob/409a08e310645f5a82234bc3a49b40c98fb98cdd/lib/docker.js#L1432

Closer than I've ever been now. But I still don't know how to use that command yet.

[Lonkle]

Or...or, what if I forced the ExposedPort to mirror the httpPort. That would solve the Nginx "502 Bad Gateway" issue (which is the last hurdle for a perfectly working PoC). I mean, is there really any reason for an exposed port in a Cloudron sense since it just reverse proxies all apps exposed ports to 80, 443 anyway. I think I'm going to try this first instead of the whole diving into dockerode thing.

[Lonkle]

Okay...there is a tiny reason. More like a medium reason. More like a BIG reason. Which is, the web apps themselves most-of-the-time choose to listen only on their exposed port (which means, I'd still need to manually proxy their exposed port to their app.httpPort...somehow). But I'll deal with that later. Gotta try this "mirror ports" experiment first to even see if it works.

Making a PoC is more important than those logistics rn tho.

[robi]

@Lonk what if you virtualized the exposed port? another nginx that you control?

[Lonkle]

@robi That's a great idea. I've thought about it before, and I'm 99% sure it would work...I just can't wrap my head around it (it's roughly what I meant in my last post when I said " I'd still need to manually proxy their exposed port to their app.httpPort...somehow").

Tbh, I think I burnt myself out because I know you're right. I mean, putting another nginx reverse proxy behind the randomized app.httpPort (say 32344) to proxy_pass to an exposed port (say 8080) would so work. My brain is just having trouble processing the concept rn (as in, where I'd need to put that code since I think it would still have to be created at the box level).

[Lonkle]

So, your idea is my next attempt at making the OpenVPN client universal.

Oh, and I need to find a way to make Cloudron expose all of OpenVPN Client's internal ports. Or at least the common ones - because the second app inherits the exposed ports of OpenVPN Client app (so the two apps would need to match up their exposed ports somehow, and a wildcard, open all ports on the OpenVPN Client's side seems like the only way to do so).

[Lonkle]

I feel like I live-blogged my entire RE and development process with this thread.

[robi]

@Lonk said in VPN tunnel for apps:

@robi That's a great idea. I've thought about it before, and I'm 99% sure it would work...I just can't wrap my head around it.

Take a break. Please A nap, walk, anything.

@Lonk said in VPN tunnel for apps:

So, your idea is my next attempt at making the OpenVPN client universal.

Right on!

Oh, and I need to find a way to make Cloudron expose all of OpenVPN Client's internal ports.
That means Nginx goes into the OpenVPN container, no?
Rules for reverse-proxying are something along the lines of:
if coming from internal IPs, do X
if coming from cloudron IP, do Y
if coming from elsewhere, do Z to the App.

[Lonkle]

@robi said in VPN tunnel for apps:

That means Nginx goes into the OpenVPN container, no?
Rules for reverse-proxying are something along the lines of:
if coming from internal IPs, do X
if coming from cloudron IP, do Y
if coming from elsewhere, do Z to the App.

I'm new to nginx, literally only using it's .conf files to try to get this working as a mere proof of concept. But so far I'm only familiar with proxy_pass for it taking incoming 80, 443 ports to pass it to the internal Docker port (and IP...I think but I don't *think this is an IP issue, though it might be).

[Lonkle]

@robi But if I wanted to expose all of it's ports, it would actually be more of a box docker.js thing with the "exposed ports" parameter being fed to gContainer.createContainer. I should be able to do that after I give my mind a lil break.

[Lonkle]

Okay, after hardcode exposing ALL internal ports of the OpenVPN container, now the last thing is adding a second nginx reverse proxy which would mean this:

My exposedPort for the Gucamole app I'm installing to connect to the OpenVPN Client app exposes 8080. It gets a Docker internal httpPort of 32455. Nginx already correctly creates a nginx proxy config from web incoming port 80 and 443 to 32455 (somehow, presumably DNS TXT records, it even gets an SSL cert).

So, I need to have nginx after that continue to proxy it to 8080. Which normally is Docker's job (with it's binding port) but when connecting two container's networks together, it doesn't do that job with the dockerode library for some Docker-y reason. I wonder...can I reverse proxy it in the same file as the original reserve proxy server. Or do I have to create a .conf and an entirely new one?

[Lonkle]

If I have the same nginx reverse proxy "listen" to it's own forwarded port (32455), then I could proxy_pass again to the real internal port and IP (8080) of Guacomole (the test app I'm now using to connect to the vpn client app)...or maybe I need another reverse proxy residing at Internal-IP-of-Guacomole:32455 to then proxy_pass again to 8080.

[robi]

The problem there is that the stuff in the container doesn't have access outside the container, so it's hard to drop updates to the host Nginx as another .conf file.

You could tell box to do it via how it already happens for new apps. Must be an API for it.

[Lonkle]

@robi I keep trying to experiment on how to shoehorn a second Ngnix proxy into this to get it all up and running 100% with no success.

Inter-communication between the two containers would be nice, but tbh I could hardcode that if I knew how to setup the second reverse nginx inside of reverseproxy.js. Which, honestly, I can't figure out how.