Cloudron Forum

update I've poked the package with a stick, added more to the README, and have run into a few interesting things about this package. s3 functionality I was able to create a bucket and put stuff in it. This seems to be a core function of an S3-compatible object server. administrative server Garage has a notion of having an administrative API at one port. I can use httpPorts to bind this port, and I can use it. For example, if I have s3.example.com as the root domain, then admin.s3.example.com can be the home for the Admin API. And, I was able to create an administrative API token, and using it, access the admin API. static websites This one is tricky. In theory, if I configure part of the garage.toml correctly: [s3_web] bind_addr = "[::]:3902" # This wants to be set dynamically in the startup. # That way, it can grab a Cloudron variable. root_domain = ".web.s3.example.com" index = "index.html" I can serve static sites out of buckets. However, this implies domain name manipulation. And, possibly, a wildcard. As in, I'd like *.web.example.com to resolve to s3.example.com, and for Garage to pick it up (internally) on port 3902. I have explored this manually (by manipulating DNS settings in Cloudflare), but even though I have configured a bucket to serve static content, I can't (yet) convince it to serve something up. While it might be that this functionality has to be sacrificed, I think it would be a nice way (if it was baked in) to manage 100% static sites. However, that would be new machinery: a way to map domains to buckets. backups I'm not convinced the backups are good. Specifically <path>/meta/db.sqlite is the metadata database for the Garage instance. This is, as far as I can tell, all of the information about where all of the files is stored. Losing this database is tantamount to losing all of the data. I think. So, making sure it backs up correctly matters. And, it is clear that updates will need to do things like garage repair and garage migrate, in the event of migrations/changes to this metadata database. Ah: Since Garage v0.9.4, you can use the garage meta snapshot --all command to take a simultaneous snapshot of the metadata database files of all your nodes. This avoids the tedious process of having to take them down one by one before upgrading. Be careful that if automatic snapshotting is enabled, Garage only keeps the last two snapshots and deletes older ones, so you might want to disable automatic snapshotting in your upgraded configuration file until you have confirmed that the upgrade ran successfully. In addition to snapshotting the metadata databases of your nodes, you should back-up at least the cluster_layout file of one of your Garage instances (this file should be the same on all nodes and you can copy it safely while Garage is running). (Emphasis mine.) So, the backup process is something I'll need to investigate further. It might be that some manual/scripted management of this database file---and dumping it---is going to be a bit of a thing in terms of having it be a robust process. (Given that Cloudron does backups before upgrades, as long as the SQLite DB is snapshotted correctly on backup, I think it will be fine.) I suspect that a cron will need to be installed for this package that---daily?---runs the snapshot command, rotates DBs, and those are part of the backup. (I have a suspicion that Cloudron packages handle this kind of thing in the start.sh scripts?) healthcheck URL The manifest assumes that the health check URL is on the main app. In this case, if I have s3.example.com and the Admin API is at admin.s3.example.com (defined in httpPorts), I want the health check URL to be https://admin.s3.example.com/health because that is where Garage put it. I don't think I can do that with the manifest as-designed. summary I think the package is off to a good start. I have questions, but most of them are described above, and I'll probably figure things out. The health check and static site subdomains, though, might not be easily solved.

Yep. There's multiple alternatives. When/if they're appropriate as a drop-in for Minio and (for packagers) as an extension, there's more than one path. Looking only at things that seem to be "live"/viable: Service URL License Notes Garage https://garagehq.deuxfleurs.fr/ GNU AGPL EU-based, compatible with many common clients, might serve as a static site/fileserver as well SeaweedFS https://github.com/seaweedfs/seaweedfs Apache 2.0 Can run as a single binary; can grow its storage area by adding to a list of paths (which would play well with Cloudron's volume mount model) VersityGW https://github.com/versity/versitygw Apache 2.0 Essentially proxies S3 straight to the filesystem, allowing access to files either through the S3 API or directly through the underlying filesystem. (Sounds easy to backup.) RustFS https://github.com/rustfs/rustfs Apache 2.0 Explicitly supports a single disk/single node deployment, but it looks like it wants direct access to disk mounts. Ozone https://github.com/apache/ozone Apache 2.0 Apache Foundation object store project. Handles HDFS and intended to scale. Not really appropriate for Cloudron's use-case. Garage and/or Seaweed are likely the most mature of this bunch. Versity might be the simplest. the start of a package After looking at the Garage repo, it was apparent that it should be very packageable. All the right things are broken out as environment variables. https://git.jadud.com/jadudm/cloudron-garage I was able to: Push this to a private registry I'm hosting on my Cloudron Use the cloudron build and cloudron update sequence to make changes and improve it on my Cloudron instance Use the terminal to create a zone/location, assign it some space, and create a bucket Create a key for that bucket Use mc (minio client) to put a file in the bucket and list the contents of the bucket There's a bunch more that would need to be done. A few thoughts, mostly random: The SQLite metadata needs to be backed up correctly. It might be that I've already done everything necessary by using localstorage and pointing it at that metadata DB. I used httpPorts to map almost all of the endpoints that are supposed to be public, but... I'm not sure I wired everything up correctly in the config. Something was right (since I could use the API), but I did not test (say) the admin API, and I did not expose the K/V database API. (Which... could be handy to expose.) The docs say httpPort is optional/not required, but the command line tools disagree. The docs should be updated. I didn't try and play with SSO, but I don't know if I have to? Or, there's a bunch to think about there. I think garage is kinda secure out-of-the-box (with no keys configured by default, etc.), but that doesn't mean I'm confident. As a result, perhaps SSO isn't necessary? I did not experiment with exposing anything as a web page. The notion that I could push to a bucket and use that as a static site server (as opposed to creating surfer instances, say) is compelling. But, I'd have to figure out how to map the domains... If this is a direction things go, I'd be glad to be a sounding board/help out. Because this is really about Garage, not Notion/etc., I'll continue commenting here: https://forum.cloudron.io/post/116584

Thanks a lot. It works!

@Botch8891 said in rsync backup reset- cloudron 9: However, when I am backing up to my new storage it still seems to be taking into account the previous backups on old provider The rsync state for each site is maintained separately. When you created a new site, it should just rsync afresh. What makes you think that it is not uploading fully? Is it because you see some files missing?

Not sure if cloning is exactly UI-friendly here. The user intention is to add another domain, which is a separate action than cloning. You'd assume you perform an action that is separate from other domains, I'd say. For backup sites, I am more with you: There it makes more sense to use the same site, but change a detail.

@luckow not sure if that is the right expectation, you have to use env vars. Password is just one or the many config vars of an addon. Addons can change because of user actions. For example, a user changes the SMTP addon address, user changes internal ports of an addon, user changes oidc scopes etc. In many PaaS setups, they will cycle passwords periodically, so hard-coding will never work even if you don't move instance in the long run. The behavior is the same in all app deployment platforms - heroku, DO app. See https://devcenter.heroku.com/articles/add-ons#config-var-values-can-change for some rationale.

Yes, this is with Porkbun DNS and it happens when I change a location of an app. It would be great if it could show the domain, so that I know where to look, and perhaps it would also be helpful if the domain tab marked that domain, as "invalid API key" is a permanent error anyway.

@webmaster we do offer direct support for businesses. We have a business plan exactly for businesses such as yours. If you are on a business plan, you can contact us at support@cloudron.io and we respond in a day.

I think all these are already pre-included by default - https://docs.cloudron.io/user-directory/#scopes-and-claims

Just got interested in this myself, seems like they use Docker Compose https://github.com/gramps-project/gramps-web/blob/main/docker-compose.yml