Garage, an open-source distributed storage service you can self-host to fullfill many needs

[girish]

@robi I mixed two topics For an addon, no UI is required. But if we package as an app, a UI is required.

[jadudm]

@girish Glad to serve.

If the goal is an addon, then...

1. I think the backup piece should be straight-forward? (I have some more info in the packaging thread about this.) Essentially, you want to make sure the SQLite DB is backed up, and then you backup the filesystem. I have questions about what happens if (say) a restore happens (e.g. what if the filesystem path changes?), but those things can be explored.
2. There is an administrative API that (once you generate a secret/trusted admin API token) lets you do everything administratively via that API (bucket creation, etc.).

As to a web interface, I would not recommend you create one for end-users. (I suspect this is not what you mean.) You have NextCloud, XBackBone, and other apps that can talk to an S3-compatible backend for file browsing. What people might need/want is a way to:

• Create/remove buckets (which, on the backend, you'd use your secret admin key)
• Create/remove keys and attach them to buckets
• Create/remove administrative keys (for superusers who want to script things against the backend)
• Bonus: the ability to designate a bucket as a static site, and then you do the DNS work on the backend to point either a subsubdomain at it (e.g. site.s3.example.com) or a whole new domain (e.g. someothersite.com -> site.s3.example.com)

I suspect you could iterate towards this, if you wanted to. Release it with terminal-only management to start, and work towards an admin interface for common bucket creation/removal-type tasks.

There are things that Garage does not do (e.g. lifecycle rules), so this is not a "full" S3 clone (the way Minio aspired to be). In this regard, SeaweedFS might offer more features (and a built-in admin UI)... so, it might be worth doing a spike to explore it as well. At a glance, it is similar, but it also is intended to scale (e.g. it does Raft consensus and Reed-Solomon EC if you want it) in a way that Garage does not. This might also be a reason to not use Seaweed.

I can poke it with a stick if it would be valuable.

Update, a few hours later: I would recommend sticking with Garage.

[timconsidine]

Purely so I could learn, I packaged Garage S3 with its WebUI module.
Please check out @jadudm package because it might be more robust, better coded than my effort.

My git repo is here :
https://git.cloudron.io/timconsidine/cloudron-garages3-ui

Prebuilt cloudron docker image : tcmbp132021/cloudron-garages3-ui:v1.0.0

Available on custom app installer https://ccai.appx.uk or your own private CCAI-PersonalEdition (remember to import the master catalogue).

EDIT : Ha! Doh! small problem, external access, please standby

EDIT #2 : web ui protected by basic auth.
rclone and MinIO client (mc) S3 client access proved.

[scooke]

@timconsidine Super cool! What specs do you recommend to run this? I might, using the free Tier, just to see how it goes, but I'll need to spin up the right VPS.

[timconsidine]

@scooke I didn't see any minimum requirements on the garage docs site

My instance is idling away at 20Mib but it is not under load. And the app is using the default 256Mb max memory limit.

Disk space - well, that all depends on your storage needs obviously.

It's a very nice app, but I don't think it is a resource hog. Check out this link (lower down the page) :

https://medium.com/@kryukz/garage-standalone-your-lightweight-s3-compatible-object-storage-journey-5073bd51b566

https://portalzine.de/day-38-garage-object-storage-the-self-hosted-s3-alternative-7-days-of-docker/

[jadudm]

@timconsidine , I'll look at Voltron-ing the two repositories together.

[timconsidine]

@jadudm I have zero idea what that means but sounds great !

[robi]

@timconsidine it's a reference to an anime cartoon where multiple parts of a robot combine into one large powerful version.

[jadudm]

This is my new favorite sequence of posts in this forum. Thank you, @robi , for the assist.

[jadudm]

@timconsidine , I'd like to look at combining your package and mine. Should https://git.cloudron.io/timconsidine/cloudron-garages3-ui be public? It just says Retry later.

[jadudm]

Nevermind. It was temporary. I retried later.