Suddenly Matrix federation doesn't work anymore :(
-
Symptom is: no new messages in channels from other instances & no search results.
In the logfiles I found
synapse.http.matrixfederationclient - 503 - INFO - POST-6883 - {GET-O-74} [matrix.org] Got response headers: 401 Unauthorized synapse.http.matrixfederationclient - 580 - WARNING - POST-6883 - {GET-O-74} [matrix.org] Request failed: GET matrix://matrix.org/_matrix/federation/v1/publicRooms?include_all_networks=false&limit=20: HttpResponseException('401: Unauthorized') synapse.http.server - 86 - INFO - POST-6883 - <XForwardedForRequest at 0x7fe7244a3160 method='POST' uri='/_matrix/client/r0/publicRooms?server=matrix.org' clientproto='HTTP/1.1' site='8008'> SynapseError: 401 - Failed to find any key to satisfy VerifyJsonRequest(server=example.org, key_ids=['1234567:abcde'], min_valid=000000000)I've tried the curl command from the Cloudron docs
$ curl https://example.com/.well-known/matrix/server
Instead of{ "m.server": "matrix-homeserver.example.com:443" }I got the following response:<html> <head><title>302 Found</title></head> <body> <center><h1>302 Found</h1></center> <hr><center>nginx</center> </body> </html>BTW: same for the cloudron.io domain.
-
@luckow Does it work with a default
homeserver.yaml? Depending on when you installed it some setting might've not been carried over to the new version. This is the first thing I try when updates break an app. -
Can you verify that the settings for the domain are correct: https://docs.cloudron.io/domains/#matrix-server-location
@nebulon settings are correct. I've rebooted the whole server instance. Nothing changed
The feedback of the federation tester (https://federationtester.matrix.org) isConnection Errors Get "https://IP.EXAMPLE.ORG:8448/_matrix/key/v2/server": context deadline exceeded (Client.Timeout exceeded while awaiting headers)And with curl
https://example.com/.well-known/matrix/serverI've got the same answer from the cloudron.io domain that my domain gives me in return. But the feedback of the federation tester is different if i use cloudron.io as the test domain.Any other ideas?
-
Ok, so I've did some testing, is it possible that you only have a wildcard DNS record for your domain? Apparently matrix federation or at least the tester needs a distinct DNS record to work.
If that is the case, setup an A record for example.com pointing to your IP and after DNS propagation the federation tester should succeed. -
Ok, so I've did some testing, is it possible that you only have a wildcard DNS record for your domain? Apparently matrix federation or at least the tester needs a distinct DNS record to work.
If that is the case, setup an A record for example.com pointing to your IP and after DNS propagation the federation tester should succeed.@nebulon good guess. You're right. Normally I use *.example.org for redirecting everything to my Cloudron. Ok. I've added an exclusive sudomain for my matrix-domain. Now I'm waiting for the DNS propagation. Let's see.
-
Symptom is: no new messages in channels from other instances & no search results.
In the logfiles I found
synapse.http.matrixfederationclient - 503 - INFO - POST-6883 - {GET-O-74} [matrix.org] Got response headers: 401 Unauthorized synapse.http.matrixfederationclient - 580 - WARNING - POST-6883 - {GET-O-74} [matrix.org] Request failed: GET matrix://matrix.org/_matrix/federation/v1/publicRooms?include_all_networks=false&limit=20: HttpResponseException('401: Unauthorized') synapse.http.server - 86 - INFO - POST-6883 - <XForwardedForRequest at 0x7fe7244a3160 method='POST' uri='/_matrix/client/r0/publicRooms?server=matrix.org' clientproto='HTTP/1.1' site='8008'> SynapseError: 401 - Failed to find any key to satisfy VerifyJsonRequest(server=example.org, key_ids=['1234567:abcde'], min_valid=000000000)I've tried the curl command from the Cloudron docs
$ curl https://example.com/.well-known/matrix/server
Instead of{ "m.server": "matrix-homeserver.example.com:443" }I got the following response:<html> <head><title>302 Found</title></head> <body> <center><h1>302 Found</h1></center> <hr><center>nginx</center> </body> </html>BTW: same for the cloudron.io domain.
@luckow said in Suddenly Matrix federation doesn't work anymore
:BTW: same for the cloudron.io domain.
$ curl -L https://cloudron.io/.well-known/matrix/server { "m.server": "matrix.cloudron.io:443" }-Lfollows redirects. -
@nebulon good guess. You're right. Normally I use *.example.org for redirecting everything to my Cloudron. Ok. I've added an exclusive sudomain for my matrix-domain. Now I'm waiting for the DNS propagation. Let's see.
-
@luckow
*.example.comdoes not coverexample.comin DNS. It only covers the subdomains
. Also, you need to have an app installed on example.comor alternately add it as a redirect of an existing app.@girish Ok. got it. From my side it was a misunderstanding. It's not a topic about "*.example.org is not enough" and to make federation work it needs to have an exclusive matrix-homeserver.example.org (btw: that makes no sense in my poor expertise about dns stuff). It the hint, that it needs a dns entry for example.org to get federation working. But: yes I have such DNS record.
-
I found the problem for my problem.
example.org is handled by the surfer app. I've switched to www.example.org this week and redirected example.org to www.example.org. That was the moment matrix federation stops working. Switching back to example.org solves the problem.
Maybe there is a bug in the surfer app @nebulon ?
Goal is to have www.example.org because of https://www.yes-www.org/why-use-www/ -
@luckow said in Suddenly Matrix federation doesn't work anymore
:BTW: same for the cloudron.io domain.
$ curl -L https://cloudron.io/.well-known/matrix/server { "m.server": "matrix.cloudron.io:443" }-Lfollows redirects.@girish should we add the -L option to the docs? https://docs.cloudron.io/apps/synapse/#step-23-verify-setup
Pronouns: he/him | Primary language: German
-
@girish should we add the -L option to the docs? https://docs.cloudron.io/apps/synapse/#step-23-verify-setup
@luckow Looks like there is a bug when the bare domain is a redirect. Investigating.
-
@girish should we add the -L option to the docs? https://docs.cloudron.io/apps/synapse/#step-23-verify-setup
@luckow I have fixed this in https://git.cloudron.io/cloudron/box/-/commit/2f58092af2344a257a05ab31f773ebbfb558eb4f
It's a small change if you want to fix it yourself. The file is
/home/yellowtent/box/src/nginxconfig.ejs. Just have to add those two lines in the commit above. Then go to Location -> Save to regenerate the nginx config. No need to restart box code. -
@luckow I have fixed this in https://git.cloudron.io/cloudron/box/-/commit/2f58092af2344a257a05ab31f773ebbfb558eb4f
It's a small change if you want to fix it yourself. The file is
/home/yellowtent/box/src/nginxconfig.ejs. Just have to add those two lines in the commit above. Then go to Location -> Save to regenerate the nginx config. No need to restart box code. -
Sorry to revive this but I had a question regarding this. It seems like Element can no longer connect when I do this. Also, I noticed the same sort of setup is here for Mastodon: https://docs.cloudron.io/domains/#matrix-server-location
So is it then possible to have the same base domain for both? I want to have matrix.domain.com and social.domain.com. Just a little confused.
-
Sorry to revive this but I had a question regarding this. It seems like Element can no longer connect when I do this. Also, I noticed the same sort of setup is here for Mastodon: https://docs.cloudron.io/domains/#matrix-server-location
So is it then possible to have the same base domain for both? I want to have matrix.domain.com and social.domain.com. Just a little confused.
@atrilahiji so the apps can be installed on any domain really but for the user/channel handles to work in federation, the base domain (in your example domain.com) needs to provide information where to find the backend servers. That information is stored in a well known location.
We've just added those cases in the domain configuration directly, to avoid users having to edit text files in specific URL paths, which can be error prone.
