Proposal: The CUR - Cloudron User Repository

[BrutalBirdie]

The Cloudron User Repository - CUR

The what?
CUR is deviated from AUR - Arch User Repository

AUR is awesome and makes it very easy for Arch Linux Users to install software from other users.

In similar fashion I would love such a feature in Cloudron.
I packaged:

• Greenlight
• Valheim Dedicated Server
• FiveM GTA V Multiplayer Server

From my 3 packaged apps only Greenlight made it into the app-store. Yet!
Why did Valheim and FiveM did not make it?
Simply put, the lack of users testing the app and reporting back.

Does Cloudron need an CUR?

Arch Linux is loved for the AUR and I am sure that CUR would also have this effect for Cloudron.
I believe many users do not test apps since they are used to the app-store and even the thought of touching the shell to install a community app with the cloudron-cli is very spooky to the targeted end-user of cloudron.
Especially on productive systems doing such thing is a no-go, since you want stable prod which does not fail you.

Now imagine a CUR.
With a simple click in the setting you can enable the CUR and can install a community app to test and give feedback.

This would decrease the barrier for end-users and @appdev(s).

This could also decrease work for maintaining apps if they stay community apps and are not suited for the general Cloudron app-store.
CUR should be excluded from Cloudron Support for obvious reasons.

I would love to get some thoughts on this.

Cheers,
Elias

(ps: I am aware of many side effects of AUR/CUR and stale unmaintained packages, but AUR still rocks )

[thetomester13]

@brutalbirdie I really like this thought! (Note, I'm not familiar at all with AUR).

This is actually why I began the Non App Store Apps repo and forum post. You're more than welcome to contribute to the repo, or send me details about the apps you packaged and I can update the forum post to reflect those.

That being said, I do think this is a nifty idea. How do you see this being built out? Maybe there's an 'official unofficial' internal Cloudron Docker where some development can occur which would make the images exposed to other devs?

[girish]

Currently, all that an end user needs is to install an app is CloudronManifest.json and the docker image. For a start, would it be better if we just provide a way to install an app using these two fields are "uploadable"? Currently, there is no easy way to point to a git(hub/lab) repo and install away since the "docker image" needs to be available somewhere. Updates would work similarly I guess.

(Making a separate repository like AUR with search etc is a lot of work...I think. Maybe we can hack something in our database to mark something as AUR, not sure)

[robi]

@girish

Can we merge the build app with the registry app so that this can happen?

Perhaps the other way would work too, add what's needed to the registry app so it can pull, build and install within it's container and then push to any cloudron from there.

[jimcavoli]

Looking the other way, there's really no reason that the Manifest can't itself specify the image repository URL, right? Then it would just be a matter of POSTing that manifest to an endpoint for the proper delivery to be possible downstream. Opens up a lot of options the more that I think about it. Sort of an elegant way to do continuous delivery automation for any custom apps as well.

[girish]

@jimcavoli indeed, that's the way it is internally.

When building an image: the CloudronManifest.json is in a git repo. the built docker image is stored in ~/.cloudron.json. When publishing to the store or installing on Cloudron these two things are merged and it's essentially just the manifest with dockerImage property.

Any ideas on how we a developer can create this merged blob so that a user can upload this merged blob? Updating the CloudronManifest.json in-place would make it appear in git diff all the time which I think is not desirable work flow.

[jimcavoli]

@girish I see what you're saying. Perhaps an ability to log into one's Cloudron.io account and post via a subcommand cloudron publish with the CLI tool would be able to send a payload of the current docker repository URL and manifest to an endpoint, which could then be sanity-checked that it doesn't require authentication, and added into the app store updates/listings (optionally with a review step), marked as community-supported. The review step is probably fairly important to see that there's a support email and such listed, but we could get additional forum topics and a process for contributing to the documentation as well set up in time.

[mehdi]

Love the idea.

My 2 cents on how this could be implemented the most easily:

• Add another "tag" on the Cloudron app database, besides "unstable". Maybe we could call it "WIP", "community", or "alpha" ?
• Re-allow normal users to publish with the Cloudron CLI, but by default (if they are not cloudron staff), their apps are tagged as "community".
• Add a setting in the Cloudron admin interface, that allows showing "community" apps in the app store. Of course, there would be proper warning alerts that tell you this is not supported & such. Maybe it could even be a bit like android developer tools, and need a secret click combination to be displayed or something, to make it clear that it's not for everyone.

I don't think this would be to much work for the staff, and I believe it would cover most, if not all, use-cases.

[girish]

I would prefer if the apps don't end up in our database at all. This is because the database tracks many things including appstore ids and I don't want to pollute our db without having vetted it. In general, we don't want to take the responsibility of any of the things in the package since we will be "hosting" it - this includes ids, screenshots, description, post install messages etc. FWIW, we already get DMCA take down messages for random screenshots (like in emby package).

Ideally, if we can figure a way by which a user can easily install an app completely outside our supervision, either by pointing to a git(hub/lab) OR maybe the user can upload some "package" file, then it's easier for us to implement this.

Of course, if there is no way out, we have to consider putting in our database...

[mehdi]

@girish Sadly it makes sense ...

Could you please list all the stuff your registry provides to the Cloudron server in order to list and install apps? The manifest, the path to the docker image, what else?

[girish]

@mehdi only the manifest and dockerImage are required and nothing else. The manifest is normalized - the about, description, postInstallMessage etc can be set as file:// in the repo but they are replaced with the file contents. The logo is currently sent separately but we can base64 encoded it or something and put it in manifest as well (we can fix this part). The screenshots are external URLs and just links inside manifest.

So.. if let's say there was a command "cloudron bundle" or something and it outputs a foo.app which is really just a JSON file which is the normalized manifest and the last built dockerImage, would that be good enough ? For the user, a user can just paste a public URL that contains the file and we can install it.

[girish]

Of course, this doesn't solve the whole "discovery" aspect of these apps. But for the moment, we can just open a forum thread that contains all the apps ( I think @thetomester13 was working on something like this? )

[murgero]

I think the ability to install via uploading a cloudron manifest (which should have an image URL) and via git (github, gitlab, other git solution) would be plenty.

Then all you would need is a "third-party" app store site for users to post on, which I can make if internal workload is too much. (not as an app but something someone can host for users to go to to see third party apps and such.

[fbartels]

Portainer has a mechanism that is called "templates". An admin can override the data source these templates are loaded from.

This way the cloudron admin could add another data source to the "App Store" on his Cloudron. Apps from that source could show in a dedicated category, potentially with a disclaimer above.

Plus points if the address would allow password protection (so that one could use an auth enabled surfer instance to host the file).

This is how the portainer template looks like:
https://github.com/portainer/templates/blob/master/templates-2.0.json

[mehdi]

@girish what kind of protocol does the communication between the repository and the Cloudron use ? I think the best approach would be to allow the admin to set additional app sources for their server. Then you could let the community self-host their own app stores. So apps installed in this way could still auto-update & stuff

[girish]

@mehdi it's only http/rest api. but making the appstore selfhostable is just a lot work, so maybe sometime in the far future. appstore.js in box repo had most of the api.

[nebulon]

While I am also a fan of AUR, I am very much aware to whom AUR and Archlinux as such is targeted. With this in mind, I want to lower expectations of a great third party app library of unsupported packages, readily discoverable. In fact what AUR is, would already be possible, just write a small script which wraps a list of git repos of app packages. Everything is already in place right now. A way to install without rebuilding the image, is actually something like a ppa in ubuntu or simply alternative package servers in Archlinux (and there are also plenty).

If your intentions are, as @BrutalBirdie initially said, to be able to allow more package testing while those packages are still under development, then I think adding support for what @girish mentioned is great. Not for technical sake, but to actually serve the purpose. It makes no sense to build something like a 1-click install way for people who cannot actually provide app package testing. (Just a slight hint to the discussions Archlinux maintainers had in the past with the AUR wrapper script developers...this was and still isn't a everyone-is-happy situation)

I understand the analogy is not to be taken 1to1 but just wanting to give some input from that perspective as well, since I do think it is a good analogy to what was suggested.

Also to get a bit more focus rather than the big picture, the mentioned examples of Valheim and GTA game servers, may actually be treated differently than more critical apps with personal or business data

[Lonkle]

Bookmarking this thread because there are def apps I'm building that feel like that wouldn't belong in the official store (like a Selenium interface that no one should ever attempt to convert to Cloudron's base image btw ), and want to see where this proposal ends up.

[robi]

One thing that was a big turnoff from Sandstorm and others is the only availability of severely out of date & unmaintained apps in their app stores.

That is a big plus for Cloudron managed apps and kudos to the @staff team here.

[girish]

So, thinking a bit more, I like the idea of "downloading" some package file that the user can upload to install an app. I guess this is very similar to android apk files. The main advantage over installing using a git repo link is that the packaging code doesn't need to be open source. App packages listed on the current App Store are all open source but just like app licensing it's not for us to enforce.

[robi]

@girish Lol, this bring us to things like App Images.

I'd still like the ability to have a private file repo and http(s) based installs from things like Surfer.

To generate the app from a git repo, there could be a bundling script like you mentioned.

Updates manual, but at some point I see feature requests to be more like package repositories with tools to scan for all available updates.

[A Former User]

@girish said in Proposal: The CUR - Cloudron User Repository:

App packages listed on the current App Store are all open source

Confluence is not open source.

[murgero]

@hillside502 said in Proposal: The CUR - Cloudron User Repository:

Confluence is not open source.

That I believe is correct however I believe they allow third-party packaging like this.

[girish]

@hillside502 said in Proposal: The CUR - Cloudron User Repository:

@girish said in Proposal: The CUR - Cloudron User Repository:

App packages listed on the current App Store are all open source

Confluence is not open source.

I should have been clearer... I meant the packaging source code itself is opensource (i.e the docker file and the tests) but not the apps themselves. We actually many apps that are not open source - emby, teamspeak, the minecraft apps, the pre-installed smtp plugin in WP (was not opensource, but i have switched it to an open one now), probably forgetting more.