CORS/Timeout error when searching for public rooms

ocelotsloth

I put this into its own thread since it seems like a different error than the other similar thread here: https://forum.cloudron.io/topic/5071/explore-public-rooms-in-matrix-fails/1

When searching for other public rooms I get this error after about a 10 second timeout period:

It seems like this is probably not a CORS issue. If I try to access that URL without a valid access token I get this response:

~
❯ http get 'https://matrix.markstenglein.com/_matrix/client/r0/publicRooms?server=matrix.org'
HTTP/1.1 401 Unauthorized
Access-Control-Allow-Headers: X-Requested-With, Content-Type, Authorization, Date
Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, must-revalidate
Connection: keep-alive
Content-Type: application/json
Date: Sat, 28 Aug 2021 16:55:05 GMT
Server: nginx
Transfer-Encoding: chunked

{
    "errcode": "M_MISSING_TOKEN",
    "error": "Missing access token"
}

If instead I make the same request with my authorization token I get a 502 Bad Gateway error.

~
❯ http get 'https://matrix.markstenglein.com/_matrix/client/r0/publicRooms?server=matrix.org' 'Authorization: Bearer TOKEN_REMOVED_FROM_THIS_POST'
HTTP/1.1 502 Bad Gateway
Connection: keep-alive
Content-Length: 1859
Content-Type: text/html
Date: Sat, 28 Aug 2021 16:56:48 GMT
ETag: "61142a4e-743"
Server: nginx

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="user-scalable=no, initial-scale=1, maximum-scale=1, minimum-scale=1, width=device-width, height=device-height" />

    <title> Cloudron App Error </title>

    <!-- Use static style as we can't include local stylesheets -->
    <style>

        html {
            height: 100%;
            width: 100%;
            padding: 0;
        }

        body {
            background-color: white;
            padding: 0;
            margin: 0;
            height: 100%;
            width: 100%;
            text-align: center;
            font-family: "Roboto","Helvetica Neue",Helvetica,Arial,sans-serif;
            line-height: 1.846;
        }

        .content {
            display: flex;
            width: 100%;
            height: 100%;
            flex-direction: column;
            justify-content: center;
        }

        footer {
            background-color: #EFEFEF;
            bottom: 0;
            width: 100%;
            color: #555;
            font-size: 12px;
            text-align: center;
            padding: 5px;
            z-index: 1000;
            position: fixed;
            opacity: .5;
            transition: all .25s;
        }

        footer:hover {
            opacity: 1;
        }

        a {
            color: #2196f3;
            text-decoration: none;
            background-color: transparent;
        }

        a:hover {
            color: #0a6ebd;
            text-decoration: underline;
        }

        </style>

</head>

<body>

<div class="content">
    <h1>&#x231B;</h1>
    <p>This app is currently not responding. Please try refreshing the page in a few minutes.</p>
</div>

<footer>
    <span class="text-muted"><a href="https://cloudron.io" target="_blank">Cloudron</a></span>
</footer>

</body>
</html>

Seems like this is either the underlying matrix server taking too long to perform federation or some other problem. Looking in the application logs all I see are these entries:

Aug 28 12:27:10 ==> Fixing permissions
Aug 28 12:27:10 ==> Starting synapse

I don't see any other logs to try and diagnose what the issue could be.

Any ideas?

nebulon

@ocelotsloth not sure if this is related, but just to have it mentioned, have you configured the federation records via the domain settings? https://docs.cloudron.io/domains/#matrix-server-location

ocelotsloth

@nebulon said in CORS/Timeout error when searching for public rooms:

have you configured the federation records via the domain settings? https://docs.cloudron.io/domains/#matrix-server-location

I have, yep. Testing the /.well-known/matrix/server path yields the correct domain as well.

nebulon

@ocelotsloth can you further look into the synapse log file at /run/synapse/homeserver.log

I will also see if we can configure synapse to log to stdout so the normal logviewer will display those bits.

nebulon

The next version will only log to stdout/stderr, for now that logfile will contain more hints.

Further, the log.config file in the configs folder can be edited via the filemanger and the loglevel there can be set to DEBUG which will be in effect after an app restart.

girish

@ocelotsloth Have you tested your server against https://federationtester.matrix.org/ for federation setup ?

girish

Actually, I am able to reproduce this when joining a public room which has > 10k users. The browser says CORS error. @ocelotsloth trying to debug more to see if this is a packaging issue or something else. But in general, it seems it has better luck with larger rooms when the memory for synapse server is increased. Right now, I have it at at 2GB and it is able to join ~8k user rooms. Can you check by bumping up the memory limit?

girish

I am able to join matrix public room now after bumping memory all the way to 8GB.

ericdrgn

@girish @ocelotsloth

I can also confirm I had this issue before increasing my RAM for Matrix. Never had an issue once I increased it. Mine is at 10gb.

nebulon

@ericdrgn wow 10gb for the matrix/synapse instance?

girish

@nebulon It seems that to join large rooms it needs much memory to start with. After joining, it seems to not need that memory. The runtime memory use is not that high.

nebulon

@girish I guess that situation then could be improved, if somehow we could have a soft memory limit to allow short temporary spikes.

ericdrgn

@nebulon Yeah, but like @girish said it isn't like it ever really hits that. And if I remember right I think I found that out via some Matrix bug and their blog posts. I think the situation has improved recently but don't think it is a Cloudron issue.

Anyway I self host so upping it to that wasn't a problem for me. Since it never mantains usage at that level I wasn't too worried about giving it that much I just got tired of random rooms still needing even more due to spikes so I just gave it a relatively absurd amount. On other instances where I have federation disabled I keep the default and have no issues.

timconsidine

@ericdrgn : I'm struggling to search and find rooms on servers with my instance set at 16Gb (just as a test).

Doesn't seem right to me that it's just a memory issue.

scooke

@timconsidine Give me a try, @scooke:incroyable.net, and I just made this room for you to try finding: #cloudron:incroyable.net

timconsidine

@scooke said in CORS/Timeout error when searching for public rooms:

#cloudron:incroyable.net

Great domain !
But no joy.
Maybe I am doing something wrong
Screenshot 2021-10-17 at 19.02.07.png

scooke

@timconsidine Hmm, you know, I recall having a similar issue when I started but I can't recal what it was. Hold on a sec...

scooke

Well, a common setting that is overlooked is having an app installed on the root domain that the Matrix server is on. So if your address is matrix.tim.com, you need to make sure there is an app (any app) installed on tim.com in Cloudron. I came across this post -

https://forum.cloudron.io/topic/3974/help-making-federation-work/31

-- which saw a user have an app installed there, and they had to go to that app's Location and re-press "Save". This is for pre-6.1 installs, but hey, you never know.

So, I guess you have Matrix and Element installed, right? Your Element is using your own Matrix?

scooke

Try https://federationtester.matrix.org/ out. It might give you a hint as to what might be missing.

timconsidine

@scooke : thank you
Yes, I have another app on the domain.
I pressed 'save' to re-save and re-start it.

The domain.tld/.well-known/matrix/server returns a response.

But the tester link you gave returns an error

Connection Errors
Get "https://107.155.122.83:8448/_matrix/key/v2/server": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

And complains there is no SRV record.
Seems to me that the tester is not compatible with Cloudron matrix installs which are on :443

Maybe it's a DNS propagation issue as it's been changed ... but has it really? because Cloudron installs are under a wildcard in my case.

I will try again in a while in case it is a propagation issue.

But there is one change.
Previously Element said "cannot find" your server incroyable.net, and now it says "you are not allowed to view this server's rooms list" and I could successfully join your room with a direct link.

And I can find some other random servers also.

Think I can this is now working, despite tester link errors.
Many thanks for your help !

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Solved CORS/Timeout error when searching for public rooms