Agate+ (dual protocol server to serve gemini/http from one source)

[timconsidine]

Full credit to :

• @DidierMalenfant for his initial package of Agate (https://code.malenfant.net/didier/agate-app), and
• Matt Brubeck for his original Agate app (https://github.com/mbrubeck/agate).

Reason for 2nd Agate package submission :

1. I can't tolerate (sorry, grumpy old fart syndrome) the separation of gemini and http worlds with different browsers, and obsessively needed to scratch my own itch, maybe others' also.
2. I don't want to add to the complexity of the http world with browser plugins or external proxies.
3. I don't want to make the package complex or have dependencies on other gemini utilities for content conversion.

So here it is : a dual protocol server, serving gemini .gmi pages natively to gemini browsers and the html equivalent to traditional browsers, by live rendering of the gemini page into html.

• single domain
• single app
• single site content source
• both worlds served

https://git.cloudron.io/timconsidine/cloudron-agate-plus

Temporary demo site (might be removed without notice) : gem.appx.uk
no protocol specified because traditional browser will supply https:// and Gemini browser will supply gemini://

I hope it helps those comfortable with building their own custom package, and wanting to enjoy the refreshing lightness of the gemini space, while not ignoring traditional web audiences.

Additional testing and feedback welcome.

Maybe @staff can review it for the cloudron App Store.
I think it represents easy maintenance, as the Agate server changes infrequently, and the package just uses the latest binary.

[sorry @LoudLemur, I didn't use the template because the package is done. ]

[timconsidine]

iOS traditional browser :

iOS DeeDum gemini browser :

[timconsidine]

Package updated to allow user-editable css of the rendered HTML page.
[gemini rendering depends on the gemini browser e.g. Lagrange being used, it's not controlled by the app]

[timconsidine]

Extra thanks to @DidierMalenfant for bringing Agate to my attention and to Cloudron.

I am excited by the potential for Agate+ to replace solutions like Ghost etc for simple sites where a CMS is overkill, without resorting to things like Jekyll and Hugo.

Those apps are great of course, but sometimes you just don't need the complexity and overhead.

Even the very wonderful Surfer app involves authoring static html by hand or in some html editing tool. With Agate+, fire up an instance of the app, open File Manager, create your Markdown content, bosh, live dual protocol website/capsule.

[timconsidine]

Clarification : when installing multiple instances of the app, slightly different CLI command is needed :
cloudron install --image docker_registry.domain.tld/cloudron-agate-plus:v2 -p GEMINI_PORT=19xx

This avoids conflict with other instances of this app.

If it makes into the App Store, user should be able to adjust the port when launching the GUI installation (obviously cannot test this at the moment).

[timconsidine]

Repo updated
Although cloudron could deploy multiple instances of the app, agate could not handle cert resolution of multiple instances hosted on the same ip address (even though different URL)
Now using an internal wildcard self-signed cert for agate, this is now supported.

I would prefer to use the letsencrypt cert, but it seems that is not possible, as it is not exposed to the app.
The public part can be captured, but the private part is hidden and not accessible.
If I am wrong on this, then maybe [@]staff or other app dev can advise me and show me how.

[robi]

How does the conversion to html happen? What serves http?

[timconsidine]

@robi I looked at some off-the-shelf utilities but for some reason I find difficult to express, I don’t like them. And become worried about dependencies.

So I wrote my own node script to read the .gmi file, which is only plain text markdown after all, and wrap the elements in basic plain html and plain css.

So the html view is generated on the fly in the app and served to a browser.
No external utilities or proxies.
Not even an internal proxy.

[robi]

Cool idea and great simple implementation. I take it your node app then serves the plain HTML/CSS.

Also a great solution for the app package liveness check since gemini runs on another port.

Two birds one stone!

[timconsidine]

@robi
yes, the node component handles the healthcheck and the ongoing rendering of the gemini source in .gmi files into clean html/css for serving to a traditional browser on an http port, while the agate component handles natively serving .gmi files to a gemini client on a tcp port (1965 unless changed by the user because of multiple installations).

It bugged me that markdown source in .gmi is so easy to render into html, so why not do it. I'm late to the gemini space, so there might be a reason, but I haven't found it yet.

I have a lurking suspicion that hard-core gemini fans almost deliberately foster the segregation of http/gemini. Seems silly to me, because the gemini project explicitly states the 2 worlds can co-exist. So let's make them co-exist.

[DidierMalenfant]

Testing it now...

• You need a chmod +x cld.sh in the repo.
• The installation never completes for me. It gets stuck at Waiting for propagation of newsite.malenfant.dev even though the DNS is updated. The app doesn't seem to launch.

Could be something with the DNS on cloudflare... Log shows:

Jul 16 17:28:25 box:dns/waitfordns resolveIp: Checking A for newsite.malenfant.dev at 162.159.38.40
Jul 16 17:28:25 box:dns/waitfordns resolveIp: No A. Checking CNAME for newsite.malenfant.dev at 162.159.38.40
Jul 16 17:28:25 box:dns/waitfordns isChangeSynced: NS davina.ns.cloudflare.com (162.159.38.40) errored when resolve newsite.malenfant.dev (A): Error: queryCname ENODATA newsite.malenfant.dev

but if I ping newsite.malenfant.dev from the command line it gets resolved correctly... odd.

Could it be that the domain is hardcoded in cld.sh?

# Generate self-signed wildcard certificate for all subdomains
echo "Generating self-signed wildcard certificate for *.appx.uk..."
# Extract the base domain (appx.uk) from CLOUDRON_APP_DOMAIN
BASE_DOMAIN="appx.uk"
WILDCARD_DOMAIN="*.${BASE_DOMAIN}"

[timconsidine]

Oh wow, big errors, will fix today
That was a test setup because of problems with self-signed certs. Should have been removed.

[LoudLemur]

@timconsidine I never thought of it as a potential Ghost replacement, but now you mention it, that is definitely one thing Agate+ could do. That dual support for html is cool!

Agate+ could also be used to plonk down when you want to park your domain or as a default page for a cloudron that is running apps on sub-domains.

[LoudLemur]

@timconsidine The geminauts are going to love your simple solution to making html available!

I tried the installation just now. and it went well up to a point. It asked where it should be installed, and then this error happened:

Port GEMINI_PORT: 19
Failed to install app: 400 message: 19 for GEMINI_PORT is not in permitted range in ports

[LoudLemur]

@timconsidine
The way this post is titled might be a bit confusing. It sounds like Agate+ is a browser that can visit gemini:// and html:// sites, rather than a gemini server which supports both gemini and html browsers.

[nebulon]

So while this app being very geeky, it seems to work rather well what @timconsidine has put together. If the individual processes within the app get run by supervisor like in other packages, to avoid just backgrounding them within start.sh I think we can add some basic integration tests and get this published. Especially since it bundles an http server it seems a much better fit for Cloudron out of the box.

Great job so far!

[LoudLemur]

@nebulon maybe test integration with goaccess?

https://goaccess.io/

[timconsidine]

@LoudLemur sorry, yes, it's broken with come code I am fixing.
Standy by please

[timconsidine]

@LoudLemur said in Agate+ (dual protocol gemini/http browser):

The way this post is titled might be a bit confusing.

err, yes, you're right - will fix somehow

[timconsidine]

@LoudLemur said in Agate+ (dual protocol gemini/http browser):

It asked where it should be installed, and then this error happened

try port 1965 ?
maybe you had a keyboard moment : 19 is not right