OIDC / routing question

[jadudm]

Hi all,

Reading through some forum threads and in particular, #11 on a thread re: OIDC from @nebulon

https://forum.cloudron.io/topic/10293/cannot-login-after-switch-to-oidc/11?_=1699652776020

I just tried to install Outline. It only does login via Cloudron OIDC. This is the first app I have tried that uses that login mechanism. It would not let me log in; it times out.

My concern is that I have a DNS issue in my home hosting setup. (I have suspected this before.) The dynamic DNS is handled by Cloudflare, I have an OpnSense firewall/router in front of Cloudron, and I wouldn't doubt that I have a configuration issue that breaks lookups for Cloudron's base hostname (e.g. my.home.net and friends) when they're handled inside my network.

Or: external requests are fine, but I have run into issues before where requests from Cloudron to Cloudron seem to fail or timeout, and I am guessing it is because of DNS resolution behind the OpnSense box.

Without asking someone to help me debug my particular configuration, I'm wondering if anyone might be able to throw me a bone as to where I should go digging... (definitely mixing the dog-chasing-bone metaphors there)... before more of Cloudron switches to OIDC, and I can't log in to anything.

Many thanks for any pointers that might spring to mind.

[girish]

@jadudm When an app is configured to use OIDC auth, it needs to be able to reach the Cloudron dashboard via https using the domain name. The requirements are:

• Should be able to reach via domain name
• Should have a valid certificate. Most apps do not work with a self signed OIDC provider.

To verify this: open a web terminal for say Outline, and in the web terminal run curl https://my.home.net . Does this work? I would go from there. You can also use host my.home.net to check what it's resolving to.

[jadudm]

Interesting.

This is all about name resolution. Now that I've started poking that with a stick...

I have Tailscale installed on the Cloudron host. It sets the DNS server entry, and further, I can use the Tailscale console to play games with DNS resolution. Before, it wasn't setting any particular external hosts. I decided to fiddle with that, and force all of my Tailscale nodes to use 1.1.1.1.

Now that I've done that, I can't get to my Cloudron instance at the my address at all. That's cool.

I now have a path to explore. I'll ping back here when I figure out what I need to do in order to resolve the my address correctly for my given home network configuration. Clearly, I'm not correctly resolving the external address to the machine internal to my network.

Thank you for the pointer. I think I've had this issue for a long time, and perhaps it is just luck that my Cloudron instance has worked for me (in my particular home network configuration) at all.

[jadudm]

I think I have a question again.

1. I have tried two things with OpnSense: using NAT reflection and DNS overrides. The DNS override for the my domain seems to work best (for machines in my internal LAN). That is, host my.cloudron returns the internal IP address.
2. My Cloudron host gets the internal IP address, and host or dig requests (or curl, etc.) all resolve nicely to the host.
3. Within the Docker containers, I get my external IP address. Or, dig my.cloudron resolves to the WAN IP address.

I tried a reboot (which didn't make any difference after upstream changes), an unbound restart (to try and flush the cache), and restarting unbound (for giggles).

I don't know why the containers are not picking up DNS changes upstream; that is, why are the containers resolving things in a manner different from the Cloudron host?

My solution was to add a new .conf to /etc/unbound/unbound.conf.d/, which feels less-than-ideal. (But, if my added file won't be removed by Cloudron, perhaps that works.

server:
    local-zone: "cloudron.domain." redirect
    local-data: "cloudron.domain. 86400 IN A 1.2.3.4"

where the domain was my Cloudron domain and the IP address the internal LAN IP of my host.

When I did this, running apps (from within the container) could now dig and curl to the my domain. However, when I tried to install something, it hangs on the propagation step. That is, an install gets most of the way there, but then hangs.

(Perhaps this is because I set an override in Unbound on the OpnSense side? I don't know. Especially because the communications to Cloudflare for DNS should be via the API... but... I admit to not understanding what is going on with the number of DNS servers that are involved. There's Unbound on the Cloudron host, Unbound on my OpnSense box, and then there's Cloudflare...)

I suspect modifying the local unbound config is not ideal, and I need to get the DNS config on my OpnSense machine right. However, I am still unclear why Docker (on the Cloudron host) is resolving names in a manner that is different from the host itself.

[jadudm]

Ah. That's why the above doesn't work.

Nov 14 09:49:54box:tasks update 3089: {"percent":85,"message":"Waiting for DNS propagation"}
Nov 14 09:49:54box:tasks update 3089: {"message":"Waiting for propagation of test1.cloudron.domain"}
Nov 14 09:49:54box:dns/cloudflare wait: test1.cloudron.domain for zone cloudron.domain of type A
Nov 14 09:49:56box:dns/waitfordns waitForDns: waiting for test1.cloudron.domain to be 123.45.67.89 in zone cloudron.domain
Nov 14 09:49:56box:dns/waitfordns Attempt 1 failed. Will retry: queryNs ENODATA cloudron.domain

I forced the local Unbound instance to use an internal LAN address (1.2.3.4), but Cloudron is expecting the resolution for the domain name to be the WAN address (123.45.67.89).

So. That didn't work.

[girish]

@jadudm the containers and host should both use unbound.

The host uses unbound via /etc/resolv.conf . It should have 127.0.0.1 . The containers will use unbound via 172.18.0.1 . You can do docker inspect <container-id> . It will have DNS server explicitly set to the bridge IP.

TBH, I am quite lost with all the updates. (Admittedly, I have no idea what tailscale is). But, if Cloudron is able to resolve my.domain.com to the WAN IP, isn't that it? Does your router not support loopback networking/hairpinning ? Is that the issue ?

[jadudm]

OpnSense should support hairpinning, but it isn't solving the problem. Or, I've explored too many dead-ends at once.

Not to worry. I'm using this thread to document at this point. I'll keep the chatter down, figure it out, and document what I learned before closing out.

[girish]

@jadudm ah no, keep putting your notes here I was just trying to understand what is going on. Maybe sometime when I sit down and read it closely, I will understand.

[jadudm]

Understood. Thank you.

I'll signal explicitly via @ if I have questions. Otherwise, consider this a documentation thread that helps us understand whether this is "user error" (as in "I have a poorly configured LAN"), or something that can improve the product. And, I'll try and capture something a bit more linear that makes clear exactly what I'm trying/doing, which would help me regardless.

This is not a source of stress for me; I do want to figure it out, but that will take the time it takes (given work, etc.).

[jadudm]

OK.

This was very exciting.

I read some documentation. Specifically, https://docs.opnsense.org/manual/how-tos/nat_reflection.html.

Once I slowed down, undid all the weird thrashing I did with various DNS shortcuts for route domains internally/directly (e.g. changing my unbound config, or creating aliases for my domain), and instead read the documentation for both reflection and hairpin NAT in OpnSense, I was good to go.

Thank you for joining me on this journey where I create noise on the forum and discover that, by reading the manual, I can solve my own problems.