Request: Include S3 Storage Module

[macaws]

Given that a lot of Cloudron servers might be set up on VMs with limited space, having the option to store on S3/S3 compatible would make a lot of sense for this app.

The official module can be found here:
https://github.com/matrix-org/synapse-s3-storage-provider

I do not believe this will add too much to the package size, and the module is maintained by the upstream project, so this addition should keep the package clean.

[girish]

There's some related interesting information in https://github.com/matrix-org/synapse/pull/7272 and https://github.com/matrix-org/synapse/blob/master/docs/media_repository.md

[girish]

@macaws It's included in the latest package.

[scooke]

@girish There it is!!

[humptydumpty]

@scooke said in Request: Include S3 Storage Module:

@girish There it is!!

Just want to point out that your domain is visible in the screenshot, in case that wasn't intended.

[scooke]

@humptydumpty Thank you. It wasn't intended, but oh well.

[ApplegateR]

@scooke hahaha, happen to me too when I am exciting to screenshot also and post it

[scooke]

@ApplegateR I think I have it set to not allow sign ups, but maybe I'll open it for Cloudronians.

[scooke]

Since I'm on a Minio/object storage kick, I tried setting this up. But it didn't work using the linked info. I tried setting it up just as path style, rather than domain style (as in, I didn't make a new A Record using the bucket name in use). I'll try later.

[nichu42]

Has anyone made this work? Care to share your steps?
I made the entries for IDrive e2 in homeserver.yaml, but I keep getting an "invalid enpoint" error:

Jan 08 13:18:56 File "/app/code/env/lib/python3.10/site-packages/botocore/endpoint.py", line 402, in create_endpoint
Jan 08 13:18:56 raise ValueError("Invalid endpoint: %s" % endpoint_url)
Jan 08 13:18:56 ValueError: Invalid endpoint: xxxx.fra.idrivee2-42.com

I double-checked with rclone: All data was entered correctly.

[nichu42]

@nichu42
OK, I found the error. The config file is expecting the endpoint URL to start with https://
I have the S3 storage up and running now. If anyone needs help, please feel free to reach out to me.

[jdaviescoates]

@nichu42 great, thanks!

If you document your process/ set-up here, you or someone else might also be able to add the details to the docs via https://git.cloudron.io/cloudron/docs

[nichu42]

@jdaviescoates

Sure! It is quite easy, once you know what you are doing.

Take the following code and replace the <S3...> boxes with your S3 compatible storage's data.

media_storage_providers:
- module: s3_storage_provider.S3StorageProviderBackend
  store_local: True
  store_remote: True
  store_synchronous: True
  config:
    bucket: <S3_BUCKET_NAME>
    region_name: <S3_REGION_NAME>
    endpoint_url: <S3_LIKE_SERVICE_ENDPOINT_URL>
    access_key_id: <S3_ACCESS_KEY_ID>
    secret_access_key: <S3_SECRET_ACCESS_KEY>

Make sure the endpoint URL starts with https://
Copy the code to your clipboard.

Open the file manager of your Synapse App in Cloudron Dashboard. Navigate to configs and open the homeserver.yaml file. Paste the code from your clipboard at the end of the configuration file.
Save and restart the app.

[jdaviescoates]

@nichu42 great, thanks!