Update on community packages

[james]

Hello @kubernetes

If you find something unclear of lacking in the documentation, please let me know so we can improve further.

[timconsidine]

@james

Spread the word

Post about new packages in the App Packaging & Development category of the forum.

Personally I think that category should remain more technical, devs needing assistance / having questions.

You published a different category : https://forum.cloudron.io/category/220/community-apps

I think that is where the word should be spread.

Just my 2p.

[james]

Hello @timconsidine
Good point.
Will update the documentation.

[timconsidine]

@james said:

If you find something unclear of lacking in the documentation ...

Just getting my head around the workflow, and I like to "spell things out" :

---

• build the Community App (previously known as a Custom App)
  (my 'old fashioned' approach : docker build, docker push, cloudron install, probably keep doing that because my build script does it, don't fix what ain't broke)

---

• make a CloudronVersions.json file
  
  same folder as project dev folder (Dockerfile, start.sh, CloudronManifest.json, README.md, POSTINSTALL.md etc.)
  • cloudron versions init
  • cloudron versions add
  • maybe add to my build script

---

• why "CloudronVersions" in the plural ?
  
  I guess Cloudron thinking is that a Community App might have v1.0.0, v1.0.1, v2.0.0
  Very complete approach, lovely
  but with cloudron versions revoke I wonder if this will ever be in true in practice (I would likely revoke every old version).

---

• upload CloudronVersions.json to static hosting
  gotcha
  but if I have 10 Custom oops Community Apps, what is Cloudron team envisioning :

  • that I will have 10 Surfer apps (app1.tim.uk, app2.tim.uk, etc) ?

  • Or 1 sectioned Surfer app (communityapps.tim.uk) ?

  • Or no Surfer apps and just stick CloudronVersions.json in the relevant git repo (urls to files from git are not always clear) ?

  • I guess you probably don't care, but I'm intrigued what your expectations are

---

Cloudron CLI help typo ?

% cloudron versions --help 
Usage: cloudron-versions [options] [command]

Hyphenated ?

---

[girish]

@timconsidine said:

upload CloudronVersions.json to static hosting

I think if the packaging code is public, it makes much sense to keep it in version control itself. If packaging code is private, you can upload it to surfer. In the case latter case, I would just have 1 surfer instance for all the apps that I would publish.

Cloudron CLI help typo ?

This is quirk of commander. Which incidentally, I just fixed yesterday - https://git.cloudron.io/platform/cloudron-cli/-/commit/a926a848400de22ddaae30f2139e0556e9461f80

[timconsidine]

@James

% cloudron versions add
Error: Bad changelog format or missing changelog for this version

'CHANGELOG' is present :

* 1.0.0
- Initial release

But I don't see any mention in cloudron docs what format is expected.

Maybe error is because this test community app has a CloudronManifest.json entry of :

"changelog": "file://CHANGELOG",

Does 'cloudron versions add' not recognise/support this way of doing it ?

---

EDIT : I changed CloudronManifest.json to read :

  "changelog": "v1.0.0 - Initial release",

Then 'cloudron versions add' worked

Not suggesting anything needs fixing as such. But potential mismatch of usage. Simple text string is easier - but it won't be practical for complex changelog descriptions.

And this cheat works :

 "changelog": "see CHANGELOG",

Not sure I should be cheating, but ...

'cloudron versions add' seems to handle 'file://POSTINSTALL.md' but not 'file://CHANGELOG'

Being creative, I tried this, thinking POSTINSTALL.md worked with file name extension.

"changelog": "file://CHANGELOG.md",

But it does not.
Seems like 'cloudron versions' treats the entries differently.

[girish]

@timconsidine I will fix the docs but the changelog file format file://CHANGELOG.md is:

[version]
* change 1
* change 2

The CLI tool reads the changelog file from the above format and then populated manifest accordingly into the versions file.

[timconsidine]

@girish
Ah, looking at output of 'cloudron versions add' in CloudronVersions.json, I see !

There is background processing/expectations, pulling only the changlog content for that version.

Neat ! But opaque pending docs clarification.

Still not sure how it will work in practice with long changelog entries, but good discipline and neat handling.

[girish]

@timconsidine it's following the same format as our existing appstore apps. For example, one of our longtime packages - https://git.cloudron.io/packages/gitlab-app/-/blob/master/CHANGELOG?ref_type=heads

[girish]

OK, updated https://docs.cloudron.io/packaging/manifest/#changelog

[timconsidine]

There's a conundrum rattling in my head.

• cloudron build prompts for a registry.
  so supplied a Cloudron-hosted registry
  and all works fine : builds, pushes, shown in registry list
• cloudron versions add does its work
• but cloudron-hosted registries do not support public pulls

So test of installing community app fails.

• am I doing something wrong ?
• or effectively cloudron versions does not support cloudron-hosted registries ?
• if so, this should be surfaced in docs
• if so, is it time to consider support for public pulls in cloudron-hosted registry package ?

Docs should also be clear that community app repo link should be open (publicly accessible), if it does not already do so.

[girish]

Right, the cloudron registry app can only host private packages. We can possibly look into adding support to make it a public registry as well, but afaik this is complicated. We need private push and public pulls - this is not implemented out of the box by the opensource registry project.

[jdaviescoates]

I think this:

@girish said:

Right, the cloudron registry app can only host private packages. We can possibly look into adding support to make it a public registry as well, but afaik this is complicated. We need private push and public pulls - this is not implemented out of the box by the opensource registry project.

Means this is correct:

@timconsidine said:

or effectively cloudron versions does not support cloudron-hosted registries ?

Right?

[girish]

@jdaviescoates Pretty much, yes.

But it can be used for sharing with friends / sharing with people you trust though. Just give them a docker registry app password and they can put it in their cloudrons and then install via the versions file.

[robi]

So it could be made public with an agreed upon default app password such as 'cloudron'

[timconsidine]

Through my CCAI --> CCAI-P --> CAG journey, I've used Docker Hub for public pulls, and my private registry for dev and non-community projects.

Maybe my bad, but I desperately hoped that Community apps would allow me one registry for minimum workflow / publish discrepancies. But seems not possible.

Not without dev work.

[made with cloudron custom mermaid app]

[robi]

It doesn't seem to make sense to separate the versions file from the source.

Since the architecture requires this for public pulls, it would make sense to have a homegrown vanilla implementation where this is taken care of.

In the case of the private registry, how could this be solved?

1. Exposing a designated source file via a local webserver at a specific path, such as .well-known/ - possibly automated by a script scanning all repos for such files, then linking them to the hosted exposed spot
2. A new addon for this purpose which accomplishes the above, which may be useful to use in other projects that need things for autodiscovery
3. Runner type workflows that work to set something like this up from the source side?
4. Cloudron provided hosting for such packages with self management capabilities?
5. Docker image hints/artifacts that can serve as a versions file???
6. Your turn..

[timconsidine]

@girish does sharing creds / creating users gives full registry access ? Or a user (virtual or real) can be limited to certain repos?

—-
EDIT : the support for Community Apps is awesome.
Thank you again for doing this “non-priority” enhancement.
Totally ok with CloudronVersions.json living in source repo such as git.cloudron.io

But plan seems to break down without easy public docker image (2 registries is good for segregating but cumbersome in practice)

[girish]

@girish does sharing creds / creating users gives full registry access ?

yes, this is why it works only for friends and trusted people.

Maybe I miss something, but cloudron docker registry app and community packages are completely different things/use cases.

• community app is for sharing built images. You generally want to use a public registry like gcs, DO, dockerhub etc. If you want to use a private registry, this is possible if the end user adds the private registry on their server. The public cloud providers have registries which give tokens with read/write and all sorts of access control which can be used for this.

• cloudron docker registry is just an app to store private docker images. sure, we can add a feature to make it support public images, but this is not on the roadmap. In general, selfhosting public docker registry is very expensive network wise and backup wise. GIven the size of docker images, you will quickly run out of network limits. This is why cloudron's own appstore images use dockerhub and is not selfhosted.

[timconsidine]

@girish you're right.
I'll have to rethink strategy.
I was focussed on simplicity, single git platform and single image registry.
But I guess it's gonna have to be separate streams :

• A: community apps : git A plus Dockerhub
• B: other dev : git B plus private image registry
  Apologies if my ambitions for combining have confused the picture/discussion.