Solved matrix.org (communication)
-
ok this is embarrassing, videochat was working all along, only my strict windows settings prohibited me from using it
-
Nice work! From a glance I can't see anything breaking when upgrading the Matrix package.
Is federation working for you though?
Log.
-
Thanks for working on this! I've tested it out and it works great so far, both Matrix and Riot.
One issue I see, and this may just be me not being very familiar with Matrix, is that when I go to search the room directory for Matrix.org, I get
Riot failed to get the public room list. Internal server error
. I don't know if I am supposed to be able to search and connect to Matrix.org rooms, but I thought I should be able to. Is this a bug or just me?Thanks again!
-
Ah nice catch, I haven't tested federation because it used to work. This could be for a number of reasons, either new regular homeserver settings, nginx config or cors related issues. I'll try to narrow it down, thanks for reporting!
-
@msbt I can test if there is a need for that.
-
@msbt I noticed this app does not actually have Riot front end? Also is the identity server implemented?
Edit: for the federation errors, here is my relevant log line(s):
2019-06-21 19:25:33,552 - synapse.http.matrixfederationclient - 433 - INFO - POST-317 - {GET-O-4} [matrix.org] Got response headers: 401 Unauthorized 2019-06-21 19:25:33,553 - synapse.http.matrixfederationclient - 517 - WARNING - POST-317 - {GET-O-4} [matrix.org] Request failed: GET matrix://matrix.org/_matrix/federation/v1/publicRooms?include_all_networks=true&limit=20: HttpResponseException("401: b'Unauthorized'",) 2019-06-21 19:25:33,554 - synapse.http.server - 112 - ERROR - POST-317 - Failed handle request via 'PublicRoomListRestServlet': <XForwardedForRequest at 0x7f13e8464ba8 method='POST' uri='/_matrix/client/r0/publicRooms?server=matrix.org' clientproto='HTTP/1.0' site=8008> Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/synapse/http/server.py", line 81, in wrapped_request_handler yield h(self, request) File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python3.6/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.6/dist-packages/synapse/http/server.py", line 316, in _async_render callback_return = yield callback(request, **kwargs) File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python3.6/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.6/dist-packages/synapse/rest/client/v1/room.py", line 387, in on_POST third_party_instance_id=third_party_instance_id, File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python3.6/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.6/dist-packages/synapse/handlers/room_list.py", line 467, in get_remote_public_room_list third_party_instance_id=third_party_instance_id, File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python3.6/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.6/dist-packages/synapse/federation/transport/client.py", line 348, in get_public_rooms ignore_backoff=True, File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python3.6/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.6/dist-packages/synapse/http/matrixfederationclient.py", line 760, in get_json timeout=timeout, File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python3.6/dist-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.6/dist-packages/synapse/http/matrixfederationclient.py", line 248, in _send_request_with_optional_trailing_slash request, **send_request_args File "/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks result = g.send(result) File "/usr/local/lib/python3.6/dist-packages/synapse/http/matrixfederationclient.py", line 472, in _send_request raise e synapse.api.errors.HttpResponseException: 401: b'Unauthorized'
-
@murgero yes, the riot frontend is here: https://git.cloudron.io/msbt/riot-app - the devs suggested not to run both on the same machine, splitting the apps was the way to go. The identity server is also not implemented (yet), so the registration uses vector.im at the moment.
Thanks for your input, I'll review the changes and try to figure out the point where the federation stopped working.
-
Also check my log records in https://forum.cloudron.io/post/3817
-
Any news on federation?
Edit: I know the devs for Cloudron recommended that these apps be separated, however I believe they would be better together (maybe a 3rd app that includes both?) because:Users can get confused on the domains (logging into riot.example.com, but user is matrix.example.com)Having them in the same place can allow for better troubleshooting, and in some cases, is just more convenient.Easier to update both at the same time, then to have to rebuild the app twice.
-
This is no recommendation from cloudron, but from the riot-devs themselves: https://github.com/vector-im/riot-web#important-security-note
I didn't have time to look further into the federation issue, maybe I'll find some time this weekend, sorry for the delay.
-
@msbt Ah my bad on misinterpreting that. and No worries. This is awesome work and worth the wait!
-
Recently learned that federation is needed for integrations to work so it's important for any integrations as well.
-
Weirdly enough, integrations are working, but federation is not... Is it possible, that the requests are being denied by cloudron and not the app @nebulon @girish? I remember we had the embedding feature which got removed and will eventually be replaced with CSP. I've tried several things, but I don't really know how to fix it... I'll jump on the matrix network and ask there if they have a clue.
-
@msbt Is there anything in the browser console? CSP/X-Frame related issue will be printed in browser console.
-
federation is working, thanks to the help of the synapse admins and community! Please grab the latest version from here and let me know if it also works for you. You might need to adjust the homeserver.yaml again, probably best if you install a fresh one and compare the config. There might be some finetuning required for preview and such, but since I'm on vacation, that's a topic for another day
-
@msbt That's weird, I installed a fresh Matrix server, still getting error 500 XHR requests when trying to federate to another room off-server. (though I can list rooms now???)
-
Alright so federation is working but some rooms don't (maybe they time out because a bridge is down??)
-
@msbt Many thanks! Canβt wait to try it.
-
I have the same issue as @murgero, I can now see federated rooms, but I get an error when trying to join them.
-
@technotame what's weirder for me is that a few rooms work.
-
I've tried several channels in both directions and it was looking good. Make sure to clear your caches and maybe give the servers more ressources regarding ram, bigger channels are very hungry.
-
@msbt yea, my container has 4GB ram. Cache was cleared and new app installed (to avoid configuration issues)
-
@murgero what's the errormessage in that case? For me it said that it can't preview the room, but joining is no problem afterwards.
-
@msbt some are "This room cannot be previewed, you must join to continue" where others are "This room is unavailable at this time"blah blah with similar output in the error logs as I mentioned above.
-
That's odd, I'm able to join any (existing) channel on the matrix network and some others as well, but those are only official networks. Maybe you guys could pm me on the chat I can try by myself. Either way, 1.1.0 got released and is already updated, maybe you could try with the new image again.
-
@msbt What is your Matrix id?
-
you can find me at @matthias:matrix.bits.at - let me know if that doesn't work
-
Hmmm it didn't work for me. I'll update with logs later when I'm home.
-
-
@technotame do you still have problems? If so, please use the federation tester to see if the target server maybe has some issues.
Also, pushed another update for riot to v1.2.4.
-
I believe it is mostly working. The link you provided gave all green checks. I think the main issue was the small amount of RAM Matrix and/or Riot was given by default. I bumped them both up to 2 gigs or so and that seemed to be the main fix.
-
@msbt Performed the upgrade. Everything seem to work as good as the old version. Need to manually edit Synapse and TURN config files to be able to report back on things like group video chat but 1on1 voice and video calls works regardless for me (at least between users on the same server)
Many thanks for your effort. Youβre effectively enabling Matrix adoption for several communities.
-
thanks @yusf, glad to hear that! maybe you can share the info if something is not working out of the box for group video chat so I can adjust the code
yes @technotame, bigger channels require a lot of ram, like the #matrix:matrix.org one with several 1000s of users, <2GB probably won't do it, maybe if you're very patient, but the more the better
-
Iβll be sure to do that. Thanks again.
-
Hi @msbt! Sadly I haven't had the time to configure and test the TURN-server.
The server has been online and stable for months btw. I'm looking to expand it with app services, webhooks for instance. However, I'm unsure on how to proceed since the Cloudron app is only configured for specific ports only and so on, limiting installing app services in Cloudron user space. Would you be interested in adding a bunch of useful default but opt-in app services?
-
hey @yusf, happy to see other people using the package
I would be happy to add more features/packages, in this case it really needs to be added in the app itself and can't be run as a regular integration. I've been thinking about including other addons as well (this seems to be an extensive setup where we could pick things that are nice2have). Let me know what might be interesting for you and I see what I can do about it.
Although it might be a good idea if either @girish or @nebulon also take a look at those and maybe give some pointers, I don't want to spend time on something which they can either do way more quickly and/or don't want in an app at all
-
I've been looking at matrix-appservice-webhooks a bit and the way it works makes it a good candidate for inclusion.
It works like so:
- a webhooks bot is created
- you invite this bot to a room
- you create a new webhook with the
!webbook
command - the bot sends you a pm with the credentials for this specific new webhook
- profit
This means that this integrations grants an unlimited amount of webhooks, so it's not a one-time thing.
Would you be willing to look into it?
-
sure thing, but I probably need you to do the testing and everything, since I don't have a use-case ready to play around with
give me a few days to free up some space and I'll look into it!
-
Thank you! While at it, check out mautrix-facebook as well, as this seems to work in a similar, multi-user fashion.
-
@msbt I can test too!
-
@msbt I can help test if needed. I use webhooks to filter by keyword and aggregate news articles, forum threads, etc.
The matrix server and riot app work great so far! Just needed a bump in memory limit when exploring the channel list from matrix.org
Recently moved from mattermost -> rocketchat and now excitedly waiting for matrix to go live in the Cloudron app store. Please let me know if there's anything I can do to help!
-
On the topic of App Store inclusion: how relevant is the attack vector of running Matrix and Riot on the same (sub)domain nowadays? I know that the Matrix folks used to recommended against that setup and perhaps they still do.
Is that threat still as relevant with the Cloudron/Docker setup? After all, Cloudron apps are supposed to work out of the box.
-
@kasini during the little time I had to try things out I didn't really get anywhere. I was having a talk with @girish a while ago and they're planning to add matrix to the app store at some point. Maybe they can have another look at it since they actually know what they're doing
And yes, it requires a lot of RAM if you want to join bigger channels, but if you keep to yourself, you should be good to go with less.
@yusf good question, if noone else does it, I'll jump on the matrix network and ask if that's still a thing to worry about
-
In addition to looking up security concerns of bundling Riot with Matrix, putting it in the app store also calls for a solution to a reverse proxy solution often used in federated software.
What I mean is a way of forwarding certain ports from
domain.tld
tomatrixserver.domain.tld
so that user handles follows convention by ommitting the technical placement of the server itself. (Hosting the server ondomain.tld
sucks for obvious reasons)
This solution would also enable more federated software with similar needs to come aboard the Cloudron ecosystem.
-
Is this app officially provided by Cloudron yet? What's the status? I see the gitlab repo but I don't know what that means.
Btw I would also love to see some bridges included as options. Bridging FB Messenger, whatsapp, telegram etc is essential if one is to use it for personal communication purposes.
-
@october As of now you have to build and install using the Cloudron CLI:
- install Cloudron CLI
- Install docker (or use cloudron build service)
git clone repo
cd repo
docker build -t dockerhubusername/projectname .
(Period is important at the end!)docker push dockerhubusername/projectname
cloudron login
cloudron install --image dockerhubusername/projectname
That's the general way to install apps not in the cloudron app store. - If using the build service provided by cloudron, replace 5 & 6 with
cloudron build
-
maubot would be a nice inclusion in the package as well. It's a bot framework, with a GUI.Not necessary to have inside this package at all. Only Application Services are!
-
Hey @msbt, the Synapse package is falling behind on releases. (1.6.0 and 1.6.1)
-
-
I looked into the possibility of a new try to host Riot and Synapse on the same (sub)domain. Hereβs the reply:
yusf:
Or is there, if itβs decided to host both on same (sub)domain, any method to reduce XSS attack probability?Riot dude:
Basically the attack surface is such that any code which gets executed with access to that subdomain in a browser will have access to that user's matrix access token. So if you run things like synapse or other things on same subdomain and they end up serving malicious code then bad things can happen.It's a very narrow surface, csp can make it even more narrow.
How then to use the CSP setting??
-
Another useful tool to possibly embed in this app package is matrix-corporal, though as an opt-in by default (enabled but void of policy) https://github.com/devture/matrix-corporal
-
just pushed an update for v1.8.0, apparently there were some changes in the config at some point, so when you're using log_file, you might need to remove that in order to be able to start the latest version.
In case it doesn't, jump on a terminal, check if it's actually running (
ps -ax
) and if not, manually launch withgosu www-data python3 -m synapse.app.homeserver --config-path homeserver.yaml
from/app/data/synapse
and check the errormessage. -
-
@msbt Can you put in a LICENSE file into the repo (preferably MIT like the other app packages), so I can get this pushed to unstable?
-
@girish Having it in unstable would be awesome.
-
@yusf in earlier versions the homeserver.yaml contained a config item
# File to write logging to. Ignored if log_config is specified. log_file: "/run/synapse/homeserver.log"
This got removed by moving it into the log_config itself which is set in
log_config: "/app/data/synapse/..."
- apparently the config for my main matrix server had that still in there because it's rather old and always migrated including this setting. Matrix wouldn't start after the latest update if you had this still enabled so I had to #comment it out be able to start afterwards.@girish sure thing, I'll look into it later today
-
-
I'm pinging for this one as well! NextGen tech on Cloudron would be great!
-
Hi there,
thanks for all your work to build an app for cloudron.
I am interested in Matrix/Synapse , too. I would like to as to put in the unstable apps section. -
@msbt Thanks for updating the license.
OK, this was stupid. I didn't see that you had a riot app, so I ended up making the front end of my own
https://git.cloudron.io/cloudron/riot-web-app . It's pretty much the same as yours except I don't use nginx (I will probably put your nginx code into mine).
I have pushed Riot Web to unstable now.
I will push out your matrix in the next few days. Thanks!