Agate+ (dual protocol server to serve gemini/http from one source)

[timconsidine]

@robi
yes, the node component handles the healthcheck and the ongoing rendering of the gemini source in .gmi files into clean html/css for serving to a traditional browser on an http port, while the agate component handles natively serving .gmi files to a gemini client on a tcp port (1965 unless changed by the user because of multiple installations).

It bugged me that markdown source in .gmi is so easy to render into html, so why not do it. I'm late to the gemini space, so there might be a reason, but I haven't found it yet.

I have a lurking suspicion that hard-core gemini fans almost deliberately foster the segregation of http/gemini. Seems silly to me, because the gemini project explicitly states the 2 worlds can co-exist. So let's make them co-exist.

[DidierMalenfant]

Testing it now...

• You need a chmod +x cld.sh in the repo.
• The installation never completes for me. It gets stuck at Waiting for propagation of newsite.malenfant.dev even though the DNS is updated. The app doesn't seem to launch.

Could be something with the DNS on cloudflare... Log shows:

Jul 16 17:28:25 box:dns/waitfordns resolveIp: Checking A for newsite.malenfant.dev at 162.159.38.40
Jul 16 17:28:25 box:dns/waitfordns resolveIp: No A. Checking CNAME for newsite.malenfant.dev at 162.159.38.40
Jul 16 17:28:25 box:dns/waitfordns isChangeSynced: NS davina.ns.cloudflare.com (162.159.38.40) errored when resolve newsite.malenfant.dev (A): Error: queryCname ENODATA newsite.malenfant.dev

but if I ping newsite.malenfant.dev from the command line it gets resolved correctly... odd.

Could it be that the domain is hardcoded in cld.sh?

# Generate self-signed wildcard certificate for all subdomains
echo "Generating self-signed wildcard certificate for *.appx.uk..."
# Extract the base domain (appx.uk) from CLOUDRON_APP_DOMAIN
BASE_DOMAIN="appx.uk"
WILDCARD_DOMAIN="*.${BASE_DOMAIN}"

[timconsidine]

Oh wow, big errors, will fix today
That was a test setup because of problems with self-signed certs. Should have been removed.

[LoudLemur]

@timconsidine I never thought of it as a potential Ghost replacement, but now you mention it, that is definitely one thing Agate+ could do. That dual support for html is cool!

Agate+ could also be used to plonk down when you want to park your domain or as a default page for a cloudron that is running apps on sub-domains.

[LoudLemur]

@timconsidine The geminauts are going to love your simple solution to making html available!

I tried the installation just now. and it went well up to a point. It asked where it should be installed, and then this error happened:

Port GEMINI_PORT: 19
Failed to install app: 400 message: 19 for GEMINI_PORT is not in permitted range in ports

[LoudLemur]

@timconsidine
The way this post is titled might be a bit confusing. It sounds like Agate+ is a browser that can visit gemini:// and html:// sites, rather than a gemini server which supports both gemini and html browsers.

[nebulon]

So while this app being very geeky, it seems to work rather well what @timconsidine has put together. If the individual processes within the app get run by supervisor like in other packages, to avoid just backgrounding them within start.sh I think we can add some basic integration tests and get this published. Especially since it bundles an http server it seems a much better fit for Cloudron out of the box.

Great job so far!

[LoudLemur]

@nebulon maybe test integration with goaccess?

https://goaccess.io/

[timconsidine]

@LoudLemur sorry, yes, it's broken with come code I am fixing.
Standy by please

[timconsidine]

@LoudLemur said in Agate+ (dual protocol gemini/http browser):

The way this post is titled might be a bit confusing.

err, yes, you're right - will fix somehow

[timconsidine]

@LoudLemur said in Agate+ (dual protocol gemini/http browser):

It asked where it should be installed, and then this error happened

try port 1965 ?
maybe you had a keyboard moment : 19 is not right

[LoudLemur]

@timconsidine thanks!

"keyboard moment" ahaha!

fails, with conflicting port. (Already have agate running, which perhaps is using port 1965. Try again though on 1966 (when England won the World Cup!) and it works... until it gives an

App installation error: Installation failed: Unable to pull image docker_registry.domain.tld/cloudron-agate-plus:v2. registry error: {"reason" : "server error"", "statusCode": 500, "json":null}

By the way, this is using the "running multiple instances" method, even though this is the first agate+ instsance.

[timconsidine]

@DidierMalenfant I fixed that hard-coding

I was struggling with self-signed certs for the scenario where a cloudron users wants to deploy multiple instances of agate+, e.g. for different sites on different subdomains. Easy for one domain, not so easy for multiple.

Cloudrono handles this quite easily, but gemini protocol is quite tight on certs, and my 2nd/3rd test instances were failing.

So I have to make a wildcard self-signed cert to handle any subdomain like xxxx.example.com.

There is still some issue, because while xxxx.example.com and zzzz.example.com will be fine under this version, probably aaaa.test.com will not be. Need to do some more testing.

Uploading fixed file to repo in a mo.

[LoudLemur]

Tried just now with a new download. we had an error:

App is being installed.

 => Queued ...
 => Deleting old containers
 => Registering location agateplus.example.com ................
 => Downloading image .....................................

App installation error: Installation failed: Unable to pull image docker_registry.domain.tld/cloudron-agate-plus:v2. registry error: {"reason":"server error","statusCode":500,"json":null}

During the course of testing this, we tried several ports (as it seemed that the default was occupied, perhaps by the previous effort, which was still using it.) We also tried a few sub-domains, as sometimes the port was accepted but the subdomain was already in use eg agateplus.example.com, agateplus2.example.com, agateplus3.example.com.

After we eventually manage to get agate+ running, how do we go about closing those ports again and stopping the "phantom" use of the sub-domains, which never worked anyway?

[timconsidine]

@LoudLemur couple of points raised :

1. cloudron only allows one assigned port like 1965 to one app. Once done, it's occupied/blocked. You need to uninstall the deployed app to clear it.
   From CLI :
   cloudron uninstall --app app.domain.tld
   Or from the app dashboard 'Uninstall' option

2. you gave a code shot which mentions "agateplus.example.com". I'm guessing you obscure the actual app URL which is fine and wise. But it also refers to a docker registry. That's an example.
   Just to clarify, did you build the app and deploy it to your registry ?
   That's necessary for custom apps, and that's probably why the 'unable to pull' message was generated.

I am still tweaking the handling of multiple deployments. Cloudron supports it, and agateplus too, but I am resolving some content caching issues.

Advice for now is stick to one deployment until I solve the cache issue.

[DidierMalenfant]

Latest update works! Thanks for the fixes.

I'm going to play with it for a bit but I'll leave a few more things I noticed in case you want to address those too:

• If you do git add --chmod=+x cld.sh locally you can then commit the executable flag with your repo and skip a step in the README.
• The cld.sh script should probably have a set -e line at the top (after the #!/bin/bash) to make sure the script stop executing if something goes wrong with one of the commands.
• Currently everything in /app/data is owned by root and I believe agate itself is launched as a root owned app. It's probably safer to make all the files in in /app/data owned by cloudron:cloudron and launch programs with /usr/local/bin/gosu cloudron:cloudron to run as the cloudron user. I might be off here but that was my understanding of recommended behavior looking at the documentation. Someone else can deny or confirm.

Great job on the app!

[timconsidine]

@DidierMalenfant Good points, thank you, will get on to them.

I'm needing to review deployment of multiple instances as it is not working 100%, but hoping to find a "cunning plan".