Amazing app to help with backups and saving space!
-
@jdaviescoates can't you just add a CNAME record that points to the S3_ALIAS_HOST ?
wrote on Nov 25, 2022, 1:39 AM last edited by jdaviescoates Nov 25, 2022, 1:40 AM@robi thanks, I think it's something to do with that yes
(from what I've read on https://thomas-leister.de/en/mastodon-s3-media-storage/ and https://chrishubbs.com/2022/11/19/hosting-a-mastodon-instance-moving-asset-storage-to-s3/ and https://github.com/cybrespace/cybrespace-meta/blob/master/s3.md )
In fact, I've done that. Here's the relevant DNS entry for safejust.space:
But that didn't seem to do the trick, I think because of cert issues which I'm not sure how to resolve.
But perhaps I'm doing something wrong?
-
@robi thanks, I think it's something to do with that yes
(from what I've read on https://thomas-leister.de/en/mastodon-s3-media-storage/ and https://chrishubbs.com/2022/11/19/hosting-a-mastodon-instance-moving-asset-storage-to-s3/ and https://github.com/cybrespace/cybrespace-meta/blob/master/s3.md )
In fact, I've done that. Here's the relevant DNS entry for safejust.space:
But that didn't seem to do the trick, I think because of cert issues which I'm not sure how to resolve.
But perhaps I'm doing something wrong?
wrote on Nov 25, 2022, 1:51 AM last edited by@jdaviescoates can you retrieve media manually via that CNAME?
-
@robi thanks, I think it's something to do with that yes
(from what I've read on https://thomas-leister.de/en/mastodon-s3-media-storage/ and https://chrishubbs.com/2022/11/19/hosting-a-mastodon-instance-moving-asset-storage-to-s3/ and https://github.com/cybrespace/cybrespace-meta/blob/master/s3.md )
In fact, I've done that. Here's the relevant DNS entry for safejust.space:
But that didn't seem to do the trick, I think because of cert issues which I'm not sure how to resolve.
But perhaps I'm doing something wrong?
wrote on Nov 25, 2022, 1:51 AM last edited by@jdaviescoates said in Amazing app to help with backups and saving space!:
But perhaps I'm doing something wrong?
Aha!
I am doing something wrong, the bucket name needs to be the same as the URL
https://www.scaleway.com/en/docs/tutorials/s3-customize-url-cname/
-
@jdaviescoates can you retrieve media manually via that CNAME?
wrote on Nov 25, 2022, 1:51 AM last edited by@robi said in Amazing app to help with backups and saving space!:
@jdaviescoates can you retrieve media manually via that CNAME?
Nope, because I need to change my bucket name...
-
@robi said in Amazing app to help with backups and saving space!:
@jdaviescoates can you retrieve media manually via that CNAME?
Nope, because I need to change my bucket name...
wrote on Nov 25, 2022, 1:52 AM last edited by@jdaviescoates yep.. media.s3...
-
@jdaviescoates yep.. media.s3...
wrote on Nov 25, 2022, 2:12 AM last edited by@robi hmz, not sure this is actually properly possible with Scaleway because at the end of this guide:
https://www.scaleway.com/en/docs/tutorials/s3-customize-url-cname/
It says:
Important:
SSL is not available when connecting to a bucket in this way.
Which I think means there is no way to resolve the certificates issue?
-
wrote on Nov 25, 2022, 7:56 AM last edited by scooke Nov 25, 2022, 7:57 AM
@scooke said in Amazing app to help with backups and saving space!:
A minio.example.com ip123
A peertube-bucket.minio.example.com ip123
A restic-backup.minio.example.com ip123
A mastodon-bucket.minio.example.com ip123I did make A records for my self-rolled Minio for the buckets to work. Without those, they were unreachable. I also had to add one more A record,
A *.minio.example.com ip123 in order for a certificate to be generated that was useful and applicable to ALL those buckets!Thats said, with this new Caprover installation, there's been no problem, no need to tweak DNS beyond the initial Caprover setup. I'm also still using path-style.
-
@robi hmz, not sure this is actually properly possible with Scaleway because at the end of this guide:
https://www.scaleway.com/en/docs/tutorials/s3-customize-url-cname/
It says:
Important:
SSL is not available when connecting to a bucket in this way.
Which I think means there is no way to resolve the certificates issue?
wrote on Nov 25, 2022, 5:40 PM last edited by@jdaviescoates said in Amazing app to help with backups and saving space!:
Which I think means there is no way to resolve the certificates issue?
you have Cloudflare right?
-
@jdaviescoates said in Amazing app to help with backups and saving space!:
Which I think means there is no way to resolve the certificates issue?
you have Cloudflare right?
wrote on Nov 26, 2022, 11:38 AM last edited by@robi said in Amazing app to help with backups and saving space!:
@jdaviescoates said in Amazing app to help with backups and saving space!:
Which I think means there is no way to resolve the certificates issue?
you have Cloudflare right?
Nope.
I dislike the centralisation caused by Cloudflare and have never really got why so many people want to give their and their uses data to them.
I think sites are many times more likely to go down due to Cloudflare outages than by getting slash dotted.
-
@scooke said in Amazing app to help with backups and saving space!:
A minio.example.com ip123
A peertube-bucket.minio.example.com ip123
A restic-backup.minio.example.com ip123
A mastodon-bucket.minio.example.com ip123I did make A records for my self-rolled Minio for the buckets to work. Without those, they were unreachable. I also had to add one more A record,
A *.minio.example.com ip123 in order for a certificate to be generated that was useful and applicable to ALL those buckets!Thats said, with this new Caprover installation, there's been no problem, no need to tweak DNS beyond the initial Caprover setup. I'm also still using path-style.
wrote on Nov 26, 2022, 12:08 PM last edited by@scooke said in Amazing app to help with backups and saving space!:
@scooke said in Amazing app to help with backups and saving space!:
A minio.example.com ip123
A peertube-bucket.minio.example.com ip123
A restic-backup.minio.example.com ip123
A mastodon-bucket.minio.example.com ip123I did make A records for my self-rolled Minio for the buckets to work. Without those, they were unreachable. I also had to add one more A record,
A *.minio.example.com ip123 in order for a certificate to be generated that was useful and applicable to ALL those buckets!Thats said, with this new Caprover installation, there's been no problem, no need to tweak DNS beyond the initial Caprover setup. I'm also still using path-style.
Thanks, that makes sense in your case where you're using a self-hosted Minio bucket on your own domains which you can control.
I'm not using Minio, I'm just using a Scaleway Object Storage Bucket, and of course I have no control over the https://s3.fr-par.scw.cloud domain and so can't fix certificate issues with that.
-
@robi said in Amazing app to help with backups and saving space!:
@jdaviescoates said in Amazing app to help with backups and saving space!:
Which I think means there is no way to resolve the certificates issue?
you have Cloudflare right?
Nope.
I dislike the centralisation caused by Cloudflare and have never really got why so many people want to give their and their uses data to them.
I think sites are many times more likely to go down due to Cloudflare outages than by getting slash dotted.
wrote on Nov 26, 2022, 12:12 PM last edited by@jdaviescoates said in Amazing app to help with backups and saving space!:
@robi said in Amazing app to help with backups and saving space!:
@jdaviescoates said in Amazing app to help with backups and saving space!:
Which I think means there is no way to resolve the certificates issue?
you have Cloudflare right?
Nope.
I dislike the centralisation caused by Cloudflare and have never really got why so many people want to give their and their uses data to them.
I think sites are many times more likely to go down due to Cloudflare outages than by getting slash dotted.
is very well regarded and might be a fine alternative to censorious, centralizing, chokepoint cloudflare.
What do others here think?
-
@jdaviescoates said in Amazing app to help with backups and saving space!:
@robi said in Amazing app to help with backups and saving space!:
@jdaviescoates said in Amazing app to help with backups and saving space!:
Which I think means there is no way to resolve the certificates issue?
you have Cloudflare right?
Nope.
I dislike the centralisation caused by Cloudflare and have never really got why so many people want to give their and their uses data to them.
I think sites are many times more likely to go down due to Cloudflare outages than by getting slash dotted.
is very well regarded and might be a fine alternative to censorious, centralizing, chokepoint cloudflare.
What do others here think?
wrote on Nov 26, 2022, 12:15 PM last edited by@LoudLemur said in Amazing app to help with backups and saving space!:
What do others here think?
IMHO, irrelevant to most people, as most people are not running sites with very high levels of traffic.
-
Here is a bit more info:
For the Minio buckets to work with Mastodon, Peertube, XBackBone, and another restic-based backup solution I set up, you MUST enter new A records for the bucket and domain of your Minio setup. I don't know how it will work on Cloudron, but for my installed-by-hand Minio instance on my KS-1, I had buckets likepeertube-bucket
,restic-backup
,mastodon-bucket
, etc. My Minio instance domain is https://minio.example.com. So, I needed to make (new) A records like:A minio.example.com ip123 A peertube-bucket.minio.example.com ip123 A restic-backup.minio.example.com ip123 A mastodon-bucket.minio.example.com ip123
After those were active, I then had to rerun
sudo certbot certonly --standalone -d minio.example.com -d peertube-bucket.minio.example.com -d restic-backup.minio.example.com -d mastodon-bucket.minio.example.com -d
and then copy the two new certs into the proper place (I imagine the Cloudron-based Minio will do all this automatically?)(Certbot calls this "Expanding" the certificate, and I actually added Expanded the two certs three times, rerunning thecertbot certonly --standalone
with all previous domains, plus whichever was the new one. It didn't work to make a new separate cert, even with it's own A Record, for, for example, resti-backup.minio.example.com plus the original minio,example.com cert. Again, I don't understand completely, but access to Minio depended on there being ONE cert with as many additional domains as necessary within it.) I tried to just use a wildcard entry for the certbot (*.minio.example.com) but it didn't work. I don't recall why, but I had to enter each sub-subdomain fully. Finally, I had to restart Minio.If you read all my previous posts asking for help, you'll see how I achieved Mino-enlightenment bit by bit, with help from others.
One thing that confused me for awhile was that I initially got into Minio with a Media Cloud plugin for Wordpress. I didn't understand at the time what
Path Style Endpoint
meant, which is the default for the Wordpress plugin Media Cloud. Basically, it worked right away just entering the Bucket name, region, and Access and Secret Keys. So I didn't realize the need for A records and SSL certs for Minio to be accessible by the other method (whose name I forget!).
I should probably rewrite all of this, make it more succinct, but for now, voila!
wrote on Oct 28, 2023, 2:00 PM last edited by@scooke said in Amazing app to help with backups and saving space!:
Here is a bit more info:
For the Minio buckets to work with Mastodon, Peertube, XBackBone, and another restic-based backup solution I set up, you MUST enter new A records for the bucket and domain of your Minio setup. I don't know how it will work on Cloudron, but for my installed-by-hand Minio instance on my KS-1, I had buckets like peertube-bucket, restic-backup, mastodon-bucket, etc. My Minio instance domain is https://minio.example.com. So, I needed to make (new) A records like:A minio.example.com ip123
A peertube-bucket.minio.example.com ip123
A restic-backup.minio.example.com ip123
A mastodon-bucket.minio.example.com ip123After those were active, I then had to rerun sudo certbot certonly --standalone -d minio.example.com -d peertube-bucket.minio.example.com -d restic-backup.minio.example.com -d mastodon-bucket.minio.example.com -d and then copy the two new certs into the proper place (I imagine the Cloudron-based Minio will do all this automatically?)(Certbot calls this "Expanding" the certificate, and I actually added Expanded the two certs three times, rerunning the certbot certonly --standalone with all previous domains, plus whichever was the new one. It didn't work to make a new separate cert, even with it's own A Record, for, for example, resti-backup.minio.example.com plus the original minio,example.com cert. Again, I don't understand completely, but access to Minio depended on there being ONE cert with as many additional domains as necessary within it.) I tried to just use a wildcard entry for the certbot (*.minio.example.com) but it didn't work.
I'm trying to use a Cloudron hosted instance of Minio to back-up some non-Cloudron hosted instances of WordPress that I help manage, and I think it's not working because of this issue.
I think perhaps the Cloudron Minio app needs to have the domain alias feature added like the WordPress (Developer) App has for use with multi-sites, so that when we have set our Minio API url to
minio-api.example.coop
and create a bucket calledbucket-name
we can also add an aliasbucket-name.minio-api.example.coop
.