Error installing Miro SFU

[MiroTalk]

Hi @nebulon, @girish

1. Purpose:

   • The purpose here is to configure MiroTalk SFU to utilize WebRTCServer for handling WebRTC connections per Worker (CPU). This can potentially optimize resource usage and improve scalability.

2. WebRTCServer:

   • WebRTCServer is a class introduced in Mediasoup version 3.10.0. It allows listening into a single port for WebRTC connections. This is advantageous for cases where multiple workers (CPUs) are involved, as each worker can have its own WebRTCServer instance.

3. MiroTalk SFU Configuration:

   • webRtcServerActive: true: This parameter indicates whether to activate the WebRTCServer. Setting it to true activates the feature.

   • webRtcServerOptions: This object holds the configuration options for the WebRTCServer.

   • listenInfos: This array contains objects specifying the listening information for the WebRTCServer.

     • Each object within listenInfos represents a listening endpoint for the WebRTCServer, specifying the protocol (UDP or TCP), IP address, and port.

     • In the provided example, both UDP and TCP protocols are used, with the same IP address (IPv4) and port number (44444) for simplicity. In practice, you might need to adjust these values based on your network setup.

4. Scaling with Workers:

   • If MiroTalk SFU starts a Worker for each CPU, each Worker will have its own WebRTCServer instance.

   • The port number for each WebRTCServer is incremented by one for each additional Worker. For instance, if you have 4 CPUs and thus 4 Workers, the ports used would be 44444, 44445, 44446, and 44447 respectively.

5. Port Range:

   • By default, each WebRTCServer internally handles the rtcMinPort and rtcMaxPort. This means that you only need to open ports for the number of Workers you have, rather than a range of ports for each Worker.

Overall, this configuration optimizes the handling of WebRTC connections in MiroTalk SFU by distributing them among multiple Workers, each with its own WebRTCServer instance listening on a specified port.

Here the corrispective configuration for MiroTalk SFU in app/src/config.js:

webRtcServerActive: true, // Set to true to activate it
webRtcServerOptions: {
    listenInfos: [
        { protocol: 'udp', ip: IPv4, port: 44444 },
        { protocol: 'tcp', ip: IPv4, port: 44444 },
    ],
},

In this configuration:

• webRtcServerActive: true activates the WebRTCServer feature.
• webRtcServerOptions contains the configuration options for the WebRTCServer.
• listenInfos specifies the listening endpoints for the WebRTCServer, including both UDP and TCP protocols on the specified IPv4 address and starting port 44444.

Here's the configuration with the additional setup for being behind a NAT:

// Behind NAT with Announced Address
webRtcServerActive: true, // Set to true to activate it
webRtcServerOptions: {
    listenInfos: [
        { protocol: 'udp', ip: '0.0.0.0', announcedAddress: IPv4, port: 44444 },
        { protocol: 'tcp', ip: '0.0.0.0', announcedAddress: IPv4, port: 44444 },
    ],
},

In this configuration:

• webRtcServerActive: true activates the WebRTCServer feature.
• webRtcServerOptions contains the configuration options for the WebRTCServer.
• listenInfos specifies the listening endpoints for the WebRTCServer, including both UDP and TCP protocols.
• ip: '0.0.0.0' indicates that the server should listen on all available network interfaces.
• announcedAddress: IPv4 specifies the announced address to be used in signaling. This is typically the public IPv4 address of the server.
• port: 44444 specifies the starting port number to listen on for both UDP and TCP protocols.

This configuration is suitable for environments where the server is behind a NAT and needs to announce its public IPv4 address for signaling purposes.

Cheers,
Miroslav Pejic

[nebulon]

@MiroTalk thanks for the elaborate description, but not sure I follow exactly. Do you suggest to not use a port range but use a specific webRtcServer feature which can handle everything on one port and also scales better or is this independent of the current config we use, which sets up a port range and configures config.mediasoup.worker.rtcMin/MaxPort? Or is both useful to have?

[MiroTalk]

@nebulon The port range configuration should still be maintained. However, with the Worker WebRTCServer feature enabled, there’s no need to manually open ports on the router. The Worker WebRTCServer handles port allocation internally. Essentially, by enabling this option, you only need to open a single port starting from 44444, with the number of ports equating to the number of server CPUs. This approach eliminates the necessity to open numerous ports per worker, streamlining the setup to just one port per worker. You can add this option as well and try if better. More about here

[nebulon]

@MiroTalk so you are saying this WebRTCServer acts similar to a reverse proxy allowing all incoming connections via one single port and internally distributes to workers locally? If this is the case, then I guess we have to change the app to only use 44444 and then the exposed (and forwarded) port range is not required?

[MiroTalk]

@nebulon More or less yes.

1. SFU Instance Deployment: When you deploy MiroTalk SFU on a server, it dynamically creates WebRtcServers based on the available CPU cores of the server if you enable this option in the config.js

2. Example CPU Core Count: For instance, if the server where you deploy MiroTalk SFU has 2 CPU cores, MiroTalk will create 2 WebRtcServers dynamically (config.mediasoup.numWorkers)

3. Port Allocation: These WebRtcServers are started from a base port number (in this case, 44444) and are internally incremented by logic for each server created.

4. Console Logs Example:

[3/26/2024, 17:37:17:682] [Server] Create a WebRtcServer {
  worker_pid: 41060,
  webRtcServerOptions: {
    listenInfos: [
      {
        protocol: 'udp',
        ip: '0.0.0.0',
        announcedAddress: 'Your-Public-IPv4',
        port: 44444
      },
      {
        protocol: 'tcp',
        ip: '0.0.0.0',
        announcedAddress: 'Your-Public-IPv4',
        port: 44444
      }
    ]
  }
}
[3/26/2024, 17:37:17:730] [Server] Create a WebRtcServer {
  worker_pid: 41061,
  webRtcServerOptions: {
    listenInfos: [
      {
        protocol: 'udp',
        ip: '0.0.0.0',
        announcedAddress: 'Your-Public-IPv4',
        port: 44445
      },
      {
        protocol: 'tcp',
        ip: '0.0.0.0',
        announcedAddress: 'Your-Public-IPv4',
        port: 44445
      }
    ]
  }
}
...

• The console logs I provided illustrate the creation of two WebRtcServers.
• Each server is associated with a unique worker process ID (worker_pid).
• Each server is configured with both UDP and TCP protocols, listening on all available IPv4 addresses (0.0.0.0) and a specific port number.
• The port numbers for each server are incremented sequentially. In this example, the first server listens on port 44444, and the second server listens on port 44445.

5. Listen Infos: Each WebRTC server's configuration (webRtcServerOptions) includes details about the protocols it supports, the IP address it listens on, the announced address (typically the public IPv4 address of your server), and the port it's listening on.

In this scenario, the application requires permission to allow traffic on ports 44444 and 44445.

[nebulon]

So on Cloudron, the firewall is actually closed down and each exposed port needs to be explicitly mentioned. Is there any configuration to limit the ports it will use?

Also I am still not sure if this is independent of the port range for the rtcMin/MaxPort configs or if this replaces it?

[IniBudi]

I also get an error message:

An error occurred during the install operation: Docker Error: (HTTP code 500) server error - driver failed programming external connectivity on endpoint 943b1ae7-08e5-4d54-8d7f-98938b449d22 (78f68ad50a0916d674692c2342ba809b82238d3e6bd7c9aacf25b67b8499821a): listen tcp4 0.0.0.0:40090: bind: address already in use

[nebulon]

There is a fix coming for the port clash issue. For the moment try to use a different port range in the app configs.

[IniBudi]

@nebulon noted, thank you!

[jdaviescoates]

@girish said in Error installing Miro SFU:

@jdaviescoates workaround is to reinstall the app with a different port range and keep your it doesn't conflict again. Note that the TCP and UDP port ranges should be the same! I recommend something like 12000 .

I tried that. Seemed to install fine, but when I actually tried to use it I get this:

I guess some of the stuff @MiroTalk has mentioned needs to be implemented before it'll work properly

[nebulon]

Please provide more detailed error logs from for example the browser console, instead of screenshots of rather generic errors. Otherwise there is really nothing to help here, unless it is easily reproducible, which for example in this case is not for me.

[jdaviescoates]

@nebulon does any of this shed any more light?

Presumably you'd also get the same issue if you visit https://miro.ud.coop (where I've just done a fresh install have have the exact same issue).

[avatar1024]

@jdaviescoates said in Error installing Miro SFU:

if you visit https://miro.ud.coop

I also get the problem a few seconds after creating and joining a room on your instance

[avatar1024]

Mine is not a fresh install and I don't get any of those issues:
https://talk.chourmo.net/
https://talksfu.chourmo.net/

[jdaviescoates]

@avatar1024 I don't have the issue on yours either.

[luckow]

same behaviour on my app instance. after a few seconds "producer transport fails" (and the console says something with WebRTC: ICE failed, add a STUN server and see about:webrtc for more details ). No problem with the instance of @avatar1024

[avatar1024]

Mine install predate the latest changes on ports stuff so I've still got the old config:

[jdaviescoates]

I just tried installing using the default 40000 ports on another Cloudron and it works fine there. I guessing changing those values just doesn't work. I wish I knew which other app on my main Cloudron was blocking those ports: is there anyway to find out?

[nebulon]

The port conflict issue is already fixed with https://git.cloudron.io/cloudron/box/-/commit/104997d77c52d39a581fa86556101357c38220a9 scheduled for next release though.

[nebulon]

@MiroTalk sorry to come back to you here, but I am trying to update the app with the port fixes, however it is not entirely clear to me how the port range (default 40000 - 40100) https://github.com/miroslavpejic85/mirotalksfu/blob/main/app/src/config.template.js#L342 and 44444+ from https://github.com/miroslavpejic85/mirotalksfu/blob/main/app/src/config.template.js#L401 is related.

Are both required and can you share what the difference is?

On top of this there is also another listenInfos at https://github.com/miroslavpejic85/mirotalksfu/blob/main/app/src/config.template.js#L408 this one does not mention any ports at all, is this correct?