Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Suddenly Matrix federation doesn't work anymore :(

    Matrix (Synapse/Element)
    5
    17
    1590
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • luckow
      luckow translator last edited by luckow

      Symptom is: no new messages in channels from other instances & no search results.

      In the logfiles I found

      synapse.http.matrixfederationclient - 503 - INFO - POST-6883 - {GET-O-74} [matrix.org] Got response headers: 401 Unauthorized
      
      synapse.http.matrixfederationclient - 580 - WARNING - POST-6883 - {GET-O-74} [matrix.org] Request failed: GET matrix://matrix.org/_matrix/federation/v1/publicRooms?include_all_networks=false&limit=20: HttpResponseException('401: Unauthorized')
      
      synapse.http.server - 86 - INFO - POST-6883 - <XForwardedForRequest at 0x7fe7244a3160 method='POST' uri='/_matrix/client/r0/publicRooms?server=matrix.org' clientproto='HTTP/1.1' site='8008'> SynapseError: 401 - Failed to find any key to satisfy VerifyJsonRequest(server=example.org, key_ids=['1234567:abcde'], min_valid=000000000)
      

      I've tried the curl command from the Cloudron docs
      $ curl https://example.com/.well-known/matrix/server
      Instead of { "m.server": "matrix-homeserver.example.com:443" } I got the following response:

      <html>
      <head><title>302 Found</title></head>
      <body>
      <center><h1>302 Found</h1></center>
      <hr><center>nginx</center>
      </body>
      </html>
      

      BTW: same for the cloudron.io domain.

      Pronouns: he/him | Primary language: German

      yusf girish 2 Replies Last reply Reply Quote 0
      • girish
        girish Staff @luckow last edited by

        @luckow I have fixed this in https://git.cloudron.io/cloudron/box/-/commit/2f58092af2344a257a05ab31f773ebbfb558eb4f

        It's a small change if you want to fix it yourself. The file is /home/yellowtent/box/src/nginxconfig.ejs. Just have to add those two lines in the commit above. Then go to Location -> Save to regenerate the nginx config. No need to restart box code.

        luckow 1 Reply Last reply Reply Quote 0
        • yusf
          yusf @luckow last edited by

          @luckow Does it work with a default homeserver.yaml? Depending on when you installed it some setting might've not been carried over to the new version. This is the first thing I try when updates break an app.

          luckow 1 Reply Last reply Reply Quote 0
          • luckow
            luckow translator @yusf last edited by

            @yusf I've installed a fresh new matrix app to a new domain. Same behavior. Maybe something is broken in the nginx configuration. (because of the wrong result in curl)

            Pronouns: he/him | Primary language: German

            1 Reply Last reply Reply Quote 0
            • nebulon
              nebulon Staff last edited by

              Can you verify that the settings for the domain are correct: https://docs.cloudron.io/domains/#matrix-server-location

              luckow 1 Reply Last reply Reply Quote 0
              • luckow
                luckow translator @nebulon last edited by

                @nebulon settings are correct. I've rebooted the whole server instance. Nothing changed 😞
                The feedback of the federation tester (https://federationtester.matrix.org) is

                Connection Errors
                Get "https://IP.EXAMPLE.ORG:8448/_matrix/key/v2/server": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
                

                And with curl https://example.com/.well-known/matrix/server I've got the same answer from the cloudron.io domain that my domain gives me in return. But the feedback of the federation tester is different if i use cloudron.io as the test domain.

                Any other ideas?

                Pronouns: he/him | Primary language: German

                1 Reply Last reply Reply Quote 0
                • nebulon
                  nebulon Staff last edited by

                  Ok, so I've did some testing, is it possible that you only have a wildcard DNS record for your domain? Apparently matrix federation or at least the tester needs a distinct DNS record to work.
                  If that is the case, setup an A record for example.com pointing to your IP and after DNS propagation the federation tester should succeed.

                  luckow 1 Reply Last reply Reply Quote 0
                  • luckow
                    luckow translator @nebulon last edited by

                    @nebulon good guess. You're right. Normally I use *.example.org for redirecting everything to my Cloudron. Ok. I've added an exclusive sudomain for my matrix-domain. Now I'm waiting for the DNS propagation. Let's see.

                    Pronouns: he/him | Primary language: German

                    girish 1 Reply Last reply Reply Quote 0
                    • girish
                      girish Staff @luckow last edited by

                      @luckow said in Suddenly Matrix federation doesn't work anymore 😞:

                      BTW: same for the cloudron.io domain.

                      $ curl -L https://cloudron.io/.well-known/matrix/server
                      { "m.server": "matrix.cloudron.io:443" }
                      

                      -L follows redirects.

                      luckow 1 Reply Last reply Reply Quote 0
                      • girish
                        girish Staff @luckow last edited by girish

                        @luckow *.example.com does not cover example.com in DNS. It only covers the subdomains 🤷 . Also, you need to have an app installed on example.com or alternately add it as a redirect of an existing app.

                        luckow 1 Reply Last reply Reply Quote 0
                        • luckow
                          luckow translator @girish last edited by

                          @girish Ok. got it. From my side it was a misunderstanding. It's not a topic about "*.example.org is not enough" and to make federation work it needs to have an exclusive matrix-homeserver.example.org (btw: that makes no sense in my poor expertise about dns stuff). It the hint, that it needs a dns entry for example.org to get federation working. But: yes I have such DNS record.

                          Pronouns: he/him | Primary language: German

                          1 Reply Last reply Reply Quote 0
                          • luckow
                            luckow translator last edited by

                            I found the problem for my problem.

                            example.org is handled by the surfer app. I've switched to www.example.org this week and redirected example.org to www.example.org. That was the moment matrix federation stops working. Switching back to example.org solves the problem.

                            Maybe there is a bug in the surfer app @nebulon ?
                            Goal is to have www.example.org because of https://www.yes-www.org/why-use-www/

                            Pronouns: he/him | Primary language: German

                            1 Reply Last reply Reply Quote 0
                            • luckow
                              luckow translator @girish last edited by

                              @girish should we add the -L option to the docs? https://docs.cloudron.io/apps/synapse/#step-23-verify-setup

                              Pronouns: he/him | Primary language: German

                              girish 2 Replies Last reply Reply Quote 1
                              • girish
                                girish Staff @luckow last edited by

                                @luckow Looks like there is a bug when the bare domain is a redirect. Investigating.

                                1 Reply Last reply Reply Quote 0
                                • girish
                                  girish Staff @luckow last edited by

                                  @luckow I have fixed this in https://git.cloudron.io/cloudron/box/-/commit/2f58092af2344a257a05ab31f773ebbfb558eb4f

                                  It's a small change if you want to fix it yourself. The file is /home/yellowtent/box/src/nginxconfig.ejs. Just have to add those two lines in the commit above. Then go to Location -> Save to regenerate the nginx config. No need to restart box code.

                                  luckow 1 Reply Last reply Reply Quote 0
                                  • luckow
                                    luckow translator @girish last edited by

                                    @girish top!

                                    Got 1 connection report.
                                    Homeserver version: Synapse 1.27.0

                                    Pronouns: he/him | Primary language: German

                                    1 Reply Last reply Reply Quote 0
                                    • ?
                                      A Former User last edited by A Former User

                                      Sorry to revive this but I had a question regarding this. It seems like Element can no longer connect when I do this. Also, I noticed the same sort of setup is here for Mastodon: https://docs.cloudron.io/domains/#matrix-server-location

                                      So is it then possible to have the same base domain for both? I want to have matrix.domain.com and social.domain.com. Just a little confused.

                                      @girish

                                      nebulon 1 Reply Last reply Reply Quote 0
                                      • nebulon
                                        nebulon Staff @Guest last edited by

                                        @atrilahiji so the apps can be installed on any domain really but for the user/channel handles to work in federation, the base domain (in your example domain.com) needs to provide information where to find the backend servers. That information is stored in a well known location.

                                        We've just added those cases in the domain configuration directly, to avoid users having to edit text files in specific URL paths, which can be error prone.

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Powered by NodeBB