Cross-app dependencies

[infogulch]

As discussed in the Expand Matrix app features thread, Matrix uses for a separate-container/out-of-process plugin/extension architecture, which clearly suggests a Cloudron implementation where each plugin is a separate app. However, this design immediately spawns some problems:

1. Flooding the app store with dozens of plugin apps. These would be pure noise for servers without Matrix already installed, and would still clutter the app list otherwise.
2. There's no straightforward mechanism to link a plugin app to a Matrix server app. Imagine a server with multiple Matrix instances installed on different domains -- which server should the plugin connect to? How would it acquire credentials for and network access to the server?

Instead of the app-per-plugin design, one could try to implement this as a single matrix integration app that has all the plugins installed and exposes options to enable/disable them as desired, but that still doesn't seem to solve #2, and would accumulate the update frequency and accidental breakage across all supported plugins. Based on this, I don't think it can be a good long-term solution.

Idea: App Dependencies

If I squint at #2, it almost seems like a generalization of the addons architecture would be the right way to conceptualize this problem: an app like Synapse could (somehow) expose a Cloudron-internal api that would fulfill a custom addon named "matrix", such that when a plugin app that has "addons": { "matrix": {} } defined in its manifest is installed, the Cloudron system would reach out to the Synapse app to fulfill some of the obligations of the addon for that app (such as providing credientials etc). With this, instead of apps requiring the Cloudron system to provide all possible dependencies, the apps could define dependencies between themselves. For example, I could imagine existing addons like postgresql being 'lifted' into this design and installed like a "normal" app (or automagically installed when it is needed like today).

This would simultaneously "flatten" the tiered approach to app dependencies with their inclusion in the regular app store (maybe in a default-hidden "dependencies" section in the store), and also enable a richer architecture for apps that have a more complex hierarchy of dependencies.

This idea also suggests a strategy to deal with the Plugin App Proliferation Problem(tm) -- since apps describe the dependencies provided by other apps inside their manifest, this gives you a natural way to begin filtering them out in different contexts.

As a second (admittedly tepid) use-case, I packaged up TerminusDB as an app because I'm mildly interested in exploring it. It has a decent built-in UI, but as its name implies it's more of a database that other apps could use as a backend than something standalone. Given the exploratory nature of my relationship with TDB, I wouldn't even request Cloudron implement it as a built-in addon to enable easy integration with test apps. However, if I could define a custom terminusdb addon I could explore both the database and potential apps that depend on it together inside a Cloudron environment.

---

Thoughts?
Have there been other discussions on this topic?

[girish]

Have there been other discussions on this topic?

I think we had a similar idea in the past, let me try to find it.

The general idea itself is sound : apps have capabilities and other apps making use of them. I remember people also wanted Android like system to expose capabilities. Anything is possible but in practice, we usually find that these extensions are usually very specific and sometimes it is way more work to write a general system than just hard coding it. For example, maybe we can just figure a way by which matrix plugins can be somehow deployed as part of the matrix app itself. We can extend the matrix package as required to accomodate the plugins. What is needed? For example, maybe a place to install plugins and also a way to "run" them (for example, drop in a supervisor file, is all that is needed?)

[girish]

I think https://git.cloudron.io/cloudron/box/-/issues/569 was what I was thinking of. It's different from yours.

[robi]

Other potentials to think about are to install the plugins in the container as a Sysbox or Chroot type isolation.

[infogulch]

we usually find that these extensions are usually very specific and sometimes it is way more work to write a general system than just hard coding it

One of the things I recognized right away about the way you guys develop Cloudron is a laser focus on implementing practical, minimal, incremental solutions instead of chasing maximum general power right out of the gate. It leads to a more stable system that I don't have to constantly fuss over -- thank you for that! I agree that this is probably the right approach for matrix plugins as well, at least for now.

I think the way issue 569 was resolved is a good example of this.

I think it makes more sense to continue the matrix discussion on that thread, which I just replied to. Thanks for taking the time to read and respond.