Docs - Alternative to Notion / Outline with OIDC, GDPR compliant, PDF Export (with template) etc...

[Teiluj]

@nebulon said in Docs - Alternative to Notion / Outline with OIDC, GDPR compliant, PDF Export (with template) etc...:

Indeed looks like they removed all the beta labels by now. Maybe I will get around looking at the package over the weekend.

This would be fantastic and much appreciated.

[jdaviescoates]

@nebulon said in Docs - Alternative to Notion / Outline with OIDC, GDPR compliant, PDF Export (with template) etc...:

Indeed looks like they removed all the beta labels by now. Maybe I will get around looking at the package over the weekend.

Just wondering... did you manage to have a look and make some progress?

[nebulon]

Took me a bit but I tried this today morning. The backend aspect is django and quite doable. Frontend unfortunately nextjs which usually blows up the resulting image by GBs, but that is also doable.

Unfortunately it does require a S3 host for media as well as keycloak for its user backend. The latter may work with our OpenID provider, however S3 is a bit of a blocker, unless we package some S3 server within the app

Will see how far we can get here over time.

[Teiluj]

@nebulon Thanks for looking into this @nebulon . Much appreciated.

[nebulon]

So I got it to the point now where oidc login works through Cloudron. If anyone is curious and maybe wants to figure out a way how to get around the S3 requirement to create any document in the first place, I pushed all package code to https://git.cloudron.io/playground/docs-app for the moment.

[andreasdueren]

@nebulon Trying to read up a bit on it. In the docs it says s3 for production is optional: "Object Storage (optional)".

Does this mean it's functional without file upload without it?

[nebulon]

Yes, however so far I couldn't figure a configuration which does not try to use S3. I did ask in the matrix channel, but so far no answer to that. Lets see...

[robi]

@nebulon why not add S3 as a default "DB" and storage layer in Cloudron?

[nebulon]

Yes, this may be worth considering, but will certainly delay getting this package out for the moment

[nebulon]

I got a response from the upstream team and indeed S3 is not optional but mandatory, the optional is for the original S3 or optionally something like minio.

[chetbaker]

hi folks, I just stumbled upon this again and it seems they just released the 4.0 version this week, including comments (!)

What an amazing piece thing would be to run this on cloudron

https://github.com/suitenumerique/docs/releases/tag/v4.0.0

[nebulon]

yes I keep an eye on that also, it seems very good. Only downside why we haven't packaged it already is the hard-dependency on S3 to store docs instead of the filesystem. With minio being in some limbo state recently this is not a great match for the moment for Cloudron, but hopefully this changes in the future

[SansGuidon]

@nebulon said in Docs - Alternative to Notion / Outline with OIDC, GDPR compliant, PDF Export (with template) etc...:

yes I keep an eye on that also, it seems very good. Only downside why we haven't packaged it already is the hard-dependency on S3 to store docs instead of the filesystem. With minio being in some limbo state recently this is not a great match for the moment for Cloudron, but hopefully this changes in the future

Minio alternative is garage: https://github.com/deuxfleurs-org/garage

[andreasdueren]

@nebulon I used to like traditional storage more but am seeing more and more the benefits of s3 in some fields. Ente performs in my opinion significantly better than immich, in good parts because of the storage used. I'm also planning on migrating various nextclouds to s3 storage. Even with MinIO not being an option anymore, there are so many vendors for people to choose for an s3 provider from. Just let people set s3 compatible credentials via s3, works great!

[robi]

@nebulon when do we get a system level S3 store?

Garage seems like a fit and enables the Multi Cloudron vision.

Let's not make us install S3 services in LAMP out of no better choice.

[jadudm]

Yep. There's multiple alternatives. When/if they're appropriate as a drop-in for Minio and (for packagers) as an extension, there's more than one path. Looking only at things that seem to be "live"/viable:

Service	URL	License	Notes
Garage	https://garagehq.deuxfleurs.fr/	GNU AGPL	EU-based, compatible with many common clients, might serve as a static site/fileserver as well
SeaweedFS	https://github.com/seaweedfs/seaweedfs	Apache 2.0	Can run as a single binary; can grow its storage area by adding to a list of paths (which would play well with Cloudron's volume mount model)
VersityGW	https://github.com/versity/versitygw	Apache 2.0	Essentially proxies S3 straight to the filesystem, allowing access to files either through the S3 API or directly through the underlying filesystem. (Sounds easy to backup.)
RustFS	https://github.com/rustfs/rustfs	Apache 2.0	Explicitly supports a single disk/single node deployment, but it looks like it wants direct access to disk mounts.
Ozone	https://github.com/apache/ozone	Apache 2.0	Apache Foundation object store project. Handles HDFS and intended to scale. Not really appropriate for Cloudron's use-case.

Garage and/or Seaweed are likely the most mature of this bunch. Versity might be the simplest.

the start of a package

After looking at the Garage repo, it was apparent that it should be very packageable. All the right things are broken out as environment variables.

https://git.jadud.com/jadudm/cloudron-garage

I was able to:

• Push this to a private registry I'm hosting on my Cloudron
• Use the cloudron build and cloudron update sequence to make changes and improve it on my Cloudron instance
• Use the terminal to create a zone/location, assign it some space, and create a bucket
• Create a key for that bucket
• Use mc (minio client) to put a file in the bucket and list the contents of the bucket

There's a bunch more that would need to be done. A few thoughts, mostly random:

• The SQLite metadata needs to be backed up correctly. It might be that I've already done everything necessary by using localstorage and pointing it at that metadata DB.
• I used httpPorts to map almost all of the endpoints that are supposed to be public, but... I'm not sure I wired everything up correctly in the config. Something was right (since I could use the API), but I did not test (say) the admin API, and I did not expose the K/V database API. (Which... could be handy to expose.)
• The docs say httpPort is optional/not required, but the command line tools disagree. The docs should be updated.
• I didn't try and play with SSO, but I don't know if I have to? Or, there's a bunch to think about there. I think garage is kinda secure out-of-the-box (with no keys configured by default, etc.), but that doesn't mean I'm confident. As a result, perhaps SSO isn't necessary?
• I did not experiment with exposing anything as a web page. The notion that I could push to a bucket and use that as a static site server (as opposed to creating surfer instances, say) is compelling. But, I'd have to figure out how to map the domains...

If this is a direction things go, I'd be glad to be a sounding board/help out.

Because this is really about Garage, not Notion/etc., I'll continue commenting here: https://forum.cloudron.io/post/116584

[jdaviescoates]

@nebulon have you had a chance to explore if we could have this on Cloudron using Garage? and/ or just by letting people connect with/ use whatever S3 service they want with it? (e.g. I think most of use are already using some external S3 service for apps like PeerTube, Mastodon, PixelFed, Ente, etc.

I think @jadudm and @timconsidine have both package Garage for Cloudron and @jadudm is in the process of trying to merge their efforts...