Can't make app VPN protection work

[bazinga]

Apologies if this is considered an app-related issue, but to me it sounds like the platform issue.

I want to restrict access to one of my Cloudron apps to a VPN only. I've got the VPN app running, client created, my OpenWRT home router has a WG interface added and a VPN routing policy added to route my desktop's requests to my Cloudron IP via the WG interface.

Doing "traceroute app.domain.net" from my home machine I see the traffic going through the VPN (packets pass 10.9.0.1, which is the default VPN network), VPN app in Cloudron shows Rx/Tx.

However the app for which I have enabled VPN Protection keeps showing "VPN connection required" page.

Not sure what exactly I am doing wrong or how to troubleshoot this. Would appreciate any help.

PS: I am on Cloudron 10.1.5.

[luckow]

@bazinga Does a direct VPN connection between your computer and your Cloudron instance work?

[bazinga]

@luckow You mean - configuring my desktop to route all traffic via the VPN? Haven't tried that because it should really be immaterial here... I've been using different kinds of VPNs for at least a decade now and running a tunnel on a router is something that I do often without any issues.

In this case I want to make sure only one domain for all home devices is ever routed via the VPN. The rest shouldn't be routed via the VPN. So running a VPN on all devices all the time isn't a solution.

[luckow]

@bazinga Yes. Just to troubleshoot - “but to me, that sounds like a problem with the platform.”
If the basic steps work, we can move on - just like with, “Did you turn it off and on again?”

[bazinga]

Setting a VPN up on the desktop is an extra annoyance that I wanted to avoid.. My thinking was - since traceroute shows traffic passing through the VPN's network that's sufficient proof of the VPN link being used.

Rebooted Cloudron. Rebooted the router couple of times (along with restarts of the firewalls and ip caches flush).

[bazinga]

Additional tests:

• Route all desktop traffic via router's WG tunnel.
  • IP / DNS leak tests on the desktop show the IP of my Cloudron.
  • The app protected by VPN Protection is not reachable and the error page is shown.
  • Review Cloudron's Event Log - for every entry it shows "Client IP" value, which is set to my Cloudron's IP - so I am accessing cloudron instance via the VPN.
• Route only Cloudron IP via policy routing on my OpenWRT router to Cloudron's VPN.
  • Can access Cloudron console and all apps that aren't configured for VPN Protection.
  • Cannot access the app being protected.
  • Review Cloudron's Event Log - for every entry it shows "Client IP" value, which is set to my Cloudron's IP - so I am accessing cloudron instance via the VPN.
• Direct access to my Cloudron's instance (no VPN)
  • Event Log entries now show my home IP under "Client IP" value
  • Cannot access the app being protected.

If this doesn't prove that something is broken with VPN Protection, then please tell me what else to check.

[bazinga]

This post is deleted!

[nebulon]

So since you mention that you see the VPN required page, this means that the IPs nginx (which does the switch to forward or show the placeholder page) config is either wrong here or it sees the wrong IPs. So from an nginx perspective it should allow requests through when the IP is the VPN app container IP.

You seem to be technical enough to maybe can evaluate the nginx configs you see with the network details you have for that Cloudron. Can you take a look over that at /home/yellowtent/platformdata/nginx/applications/<appid>/<appdomain>/conf where the app here is the one which is protected.

[bazinga]

@nebulon So, what exactly should I be looking for in the .conf file there?

When switching VPN Protection on I see at least the following to be added into the conf file:

allow 172.18.16.137;
allow fd00:c107:d509::a;
allow 172.18.16.147;
allow fd00:c107:d509::c;
deny all;
error_page 403 /vpn_required.html;

Plus few other minor changes.

docker inspect ... command returns 172.18.16.147 as an IP for cloudron/io.cloudron.openvpn container. This IP is in the above allow list. Not sure what else to check in the file?

However just thinking this through - after the VPN is traffic routed via a public DNS or internally without going back out on the public internet (eg. my desktop -> Cloudron hosted VPN app -> public internet -> VPN protected app on Cloudron OR my desktop -> Cloudron hosted VPN app -> internal network / local IPs -> VPN protected app on Cloudron)? If it's the former, then I'd expect an "allow" rule for Cloudron's IP.

[nebulon]

well then the configs look correct and the vpn app also seems to have the right IP. So no clue really why your requests then are not going via the VPN as you mentioned earlier. Maybe for good measure restart nginx just to be sure that this for some reason didn't apply.

[bazinga]

I've restarted Cloudron, which restarts everything, so I presume nginx would be ok there too. But I've also restarted nginx multiple times with no effect. Nginx logs show me connecting from Cloudron's IP.

Like I said - when I connect via the VPN even Cloudron's Event Log shows my IP as Cloudron's IP. So highly likely something is broken internally in traffic routing in Cloudron.

How can this be resolved / fixed? What kind of information do you need to troubleshoot this?

With all of these 0day issues I want to make bunch of apps closed from the Internet for the time being, so getting this VPN protection would be a good way to go.

[bazinga]

Well, as soon as I manually edit .conf file and add allow rule for the public IP of Cloudron server - I can access the app protected by VPN Protection. So, something is either broken in network routing (if the expectation for the routing to be done via internal IPs) or the .conf file should include allow PUBLIC-IP rule.

[nebulon]

Ok so if public IP works and this is what nginx sees for the app, this means you are no going via the VPN as if that would be the case nginx would see the local VPN container IP. So this is then the routing discrepancy. Basically for some reason your system reaches the server on its normal public IP and not via the tunnel.

[bazinga]

@nebulon Public IP of the Cloudron server - so I am going through the VPN because my IP is reported as the server's IP.

What happens here - my traffic is routed via the VPN, it gets routed out to the public internet (so I have server's IP) and then it hits the app being protected from the outside (not internally).

My server IP is : 1.2.3.4, my home IP is : 99.99.99.99, I have to add the following to .conf file to access the app when "VPN protection" is turned on:

allow 1.2.3.4;

And like I said before - even when all of my traffic is routed via the VPN (so that "what is my IP" and "dns leak test" report server's IP as my IP) the app still gives me the error page.

[nebulon]

So then lets do one more test. If you run host <protected app domain> while being disconnected from the VPN, you should see the server's public IP. Then connect to the VPN and run the host command once more and then you should see the VPN network so for exampls 100.64.0.1

[bazinga]

In both cases the command returns server's public IP.

I am routing all desktop's traffic through the VPN tunnel on my router - every "what is my IP" page I can find online shows my server's public IP, logging into my Nextcloud (on same Cloudron instance) resulted in an email notification saying I've logged in from a new location (with the server's IP address), Event Log entries in Cloudron show server's IP address in "Source IP" attribute.

[bazinga]

This post is deleted!

[bazinga]

This post is deleted!

[nebulon]

@bazinga yes those "whats my ip" pages are correct but that is not the issue. Externally your connection would have your server's public IP.

However, for the VPN protected app on your Cloudron they have to go locally directly between the containers on your server itself, so the public server IP is not the incoming one. Since the host commands in both cases return the public IP, there is something wrong with the DNS resolving. I don't know your exact setup there, but when connecting to the VPN it would change the IPs returned for VPN protected domains to have the 100.64.* ones to the connection does exactly not go via the public route.

Do you have any special or custom DNS configs set on your laptop or a custom DNS server which overwrites your domains?

[bazinga]

@nebulon Thanks for getting back to me yet again. Nothing custom, I do use DoT on my OpenWRT router, but that's all.

With that said, after many hours of debugging and trying many different configurations, I seem to finally be able to access VPN protected apps without any hacks. Took me a while to discover 100.64.* subnet along with other things.

The issue occurs because Cloudron’s per-app VPN protection verifies access by evaluating the client's source IP at the Nginx reverse-proxy level, which breaks when a central network router acts as a gateway and enforces NAT masquerading (Double-NAT).

The solution that worked for me is to implement Split-Horizon DNS at the router level, forwarding the app's domain queries directly to Cloudron’s internal VPN DNS relay (seems to be 10.9.0.1 in my case), which allows traffic to ingress securely via the native WireGuard network interface layer without tampering with the original client packet structure. To make this work seamlessly, users must also ensure individual client devices do not bypass the router's DNS via encrypted Secure DNS features (DNS-over-HTTPS).

Below I'll share the steps required to setup a router so that multiple devices are able to access these protected apps without needing to have VPN connection on each.

One question: why did you guys opt for this VPN protection vs simply allowing users to define a white list of source IPs or domains from where the access would be allowed? Would be much simpler and not require the VPN hassle (no matter on an endpoint or a router).