Networking between applications?

[jadudm]

I wrote a small go app to do the following:

1. Subscribe to some MQTT channels.
2. Post data that it sees to a Directus instance.

When I run the application from my dev machine, everything is fine.

When I containerize the application and run it on Cloudron, I get one dial tcp error, and then on subsequent runs (the container is automatically restarted, it seems) I get

context deadline exceeded (Client.Timeout exceeded while awaiting headers)

on my GET against the Directus server. (I did try and set a 5, 30, and 60s timeout on my calling HTTP context, none of which make a difference.)

The containerized go application is being installed to my Cloudron server, and the Directus instance is also running on my Cloudron server.

It really is a simple app. I would expect I could containerize and run it on Cloudron, given all it does is 1) listen to one server and 2) read/write to another (via HTTPS GET/POST).

Thoughts welcome.

[jadudm]

@jadudm Oh... perhaps this is a certs issue...

[jadudm]

@jadudm So.

When I use cloudron/base:3.1.0 for my container, I can get a shell (meaning, I can pound the terminal button in Cloudron and get a root prompt in the container). If, from within the container, I curl something like https://www.google.com/, I see a good grab. I can also use openssl to get the cert chain for Google. This tells me it is not a "general" SSL problem from within the container.

When I try and hit a domain like my.cloudron.example running on my Cloudron host, both curl and openssl hang.

Is this:

1. A Docker networking thing?
2. A Cloudron-does-things-to-iptables thing?
3. A my-home-opnsense-firewall-is-doing-something thing?

This is almost certainly either a 1) Cloudron/Docker network thing, or 2) an interplay with my home network setup. That said, the container is on the host it is trying to reach.

root@5b7bf2bf-f81d-4fdc-8952-13a99497d6ef:/# curl -vvv -4 https://my.cloudron.example/ 
*   Trying CORRECTIP:443...
* TCP_NODELAY set

It is doing what it should from within the container. Something in the surrounding network universe is not?

Help or recommendations would be welcome.

[robi]

@jadudm You can't treat containerized apps on the same system as an app external to the system. The network flow is different.

Cloudron Apps have their own private network, and all requests to them is proxied from outside to the inside.

While that works while your custom app is on the outside, that same pattern doesn't work from the inside.

Find the internal IP of your Directus app via ifconfig and use that to reach it from another app on the inside network.

Also remember that docker maps ports from internal to external, so you'll need to compensate for that too by knowing which ports to use internally, but now you should have enough info to connect.

[jadudm]

@robi Thank you. This makes sense.

Unfortunately, it is fragile. I am not guaranteed that my Directus instance will keep the same 172.* address. I'm trying to read from my MQTT broker (running on my Cloudron host) and insert data into my Directus server (running on my Cloudron host). Those services have good (subdomain) names in the world, and I don't want to author an application that is fragile based on dynamic, internal Docker network IP addresses in the 172 range. Is there a better way?

@robi, @girish , @nebulon, is there a more robust way for me to write an application that will run on my host and talk to another application on the host? For example, can I use the docker addon to list containers on the internal network, and from that, get their IP addresses and exposed ports?

I feel like this is information that the Cloudron runtime knows, but I as an application author do not (but might want to know).

My alternative... I can just run this on another VM on my Proxmox host. But, then I'm a sysadmin again. I really like the idea of being able to have a container that I can update and, when I do, Cloudron will pull the image and update the running application for me. (And, yes... I could use watchtower on the VM for this...)

[jadudm]

@jadudm My initial experiment suggests that I cannot use the Golang Docker SDK to do something like:

	ctx := context.Background()
	cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
	if err != nil {
		panic(err)
	}

	containers, err := cli.ContainerList(ctx, types.ContainerListOptions{})
	if err != nil {
		panic(err)
	}

	for _, container := range containers {
		fmt.Println("CID", container.ID)
	}

The expected error of

Apr 10 23:26:29 panic: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

comes up. This is with the docker addon.

It's late; I've done all the investigation I can for a weekend. If there's a solution where I can do this robustly on my Cloudron host, that would be ideal.

[girish]

@jadudm said in Networking between applications?:

A my-home-opnsense-firewall-is-doing-something thing?

The curl is supposed to work from an app shell/terminal. It's not working most likely because your network/router does not support hairpin routing.

To debug, I would first SSH into the server and check if curl -vvv -4 https://my.cloudron.example/ works i.e forget about docker etc.