Plausible (Analytics Platform)

[timconsidine]

@d19dotca I was a fan of Plausible, but to be honest, I have gone off it.

I have moved to Umami which doesn't suffer the blocking which Plausible was. I.e. Plausible tracking script gets blocked MUCH more than Umami, so it effectively under-reports traffic. (Mind you Google Analytics also does).

Vote up Umami instead is my recommendation.
But no problem to have Umami, Plausible and Ackee as options.

[d19dotca]

@timconsidine I voted up both already. But my understanding is that Umami doesn’t yet do reporting which is critical for my customers (they pay for it) so I personally would be required to go with Plausible until Umami picks up some more features like reporting capabilities. If I’ve missed that in Umami though please point me to the right doc.

[timconsidine]

@d19dotca I'm not anti-Plausible, just prefer best data.
Good to have a choice of systsems.
What type of reporting is needed?

[dylightful]

We need a better Analytics tool for Cloudron. Been using Plausible with Elest.io and its great!

[jdaviescoates]

@dylightful said in Plausible (Analytics Platform):

Elest.io

I'm thinking we ought to try and assemble a comprehensive list of Cloudron-like services like this. Seems there are so many these days! (and together it seems like at least someone here has tried them out)

[nebulon]

@jdaviescoates indeed such a list would be very useful. Although Elest.io is really in the category of providing hosted versions of OpenSource apps. So from a deployment and data ownership perspective quite different.

[jdaviescoates]

@nebulon said in Plausible (Analytics Platform):

from a deployment and data ownership perspective quite different.

Yeah, they're all a bit different in their models. For me the inclusion criteria would just be something like "makes it really easy for relatively non technical people to install a bunch of open source apps and access them on their own domains", and possibly also those that are aimed at people who are a bit more technical but also automate much of the process too.

[girish]

@dylightful Is the pricing per VM? Can multiple apps be deployed in a single VM?

[ajtatum]

@girish Sorry to awaken an old thread, but just wanted to give another vote for Plausible. I'm trying to have as few servers as possible, with the servers either being specifically for one or two specific apps or something like Cloudron (my Truenas Scale has a ton of apps, Synology NAS, and Ubuntu with Portainer). For business, looking at this, SerpBear, and Odoo. TrueNas has Odoo but for the other two it looks like I can either try Elest.io or spin up yet another server but try out Easy Panel. I've tried CapRover and had mixed experiences, and they also don't have SerpBear.

[ajtatum]

I have this running off my TrueNAS Scale and it works perfectly. I'd really prefer to not have this running there as it's the only app installed that's exposed to the internet (rest are internal apps), so this would be nice to have on Cloudron... and I definitely prefer it over the current selection of analytic solutions.

[andreas]

@ajtatum Would be great to have the app within Cloudron. Right now it's self-managed by ourselves with Portainer.

[d19dotca]

Definitely hoping to see Plausible too. Currently testing Umami but not thrilled with it in all honesty, some benefits to Plausible over Umami such as concatenated referrers and more (i.e. "Google" as opposed to tracking google.ca and google.com separately, which may be helpful to some but frankly most customers just care how much comes from "Google" overall rather than the TLD of it).

[andreas]

@d19dotca Same here. We're currently using it self-hosted, but I would prefer to shift it to Cloudron, because it's less time to maintain and a harmonized concept for all the stuff around.

[emilienbidet]

I would like to contribute by creating this app. Where should I start to learn to encapsulate app in Cloudron?

[robi]

@emilienbidet start with docs.cloudron.io

[jdaviescoates]

@robi said in Plausible (Analytics Platform):

@emilienbidet start with docs.cloudron.io

More specifically https://docs.cloudron.io/packaging/tutorial/

[girish]

@emilienbidet We have a forum section (https://forum.cloudron.io/category/96/app-packaging-development) for this. See the pinned topics there for documentation and pointers. Feel free to post there for any packaging help.

[crazybrad]

Plausible is a great platform. We have contributed some (open source) code enhancements and I can say that the two individuals behind Plausible are awesome to work with. We are hosting this on our own infrastructure but continue to maintain a subscription at Plausible.io to support the platform (Note: this is an easy and inexpensive way to test the technology). For those considering "packaging" it, there are multiple docker containers required (5 if my memory is correct on this): database (PostGRES), events (Clickhouse), email, application, geoIP database. And Plausible is GPDR-compliant!

[Sydney]

Plausible Analytics Packaging Attempt

I've began an attempt at packaging Plausible for the Cloudron platform. So far, I've spent two days on it, and it's a non-trivial effort. I'll leave some notes here for those who may be considering packaging this, or similar apps:

Overview of Plausible's architecture

Plausible depends upon two databases:

• PostgreSQL, which is provided by Cloudron's Postgres addon
• Clickhouse, a high-performance DBMS commonly used in analytics.

Clickhouse must be made available to Plausible in some way, and the upstream maintainers at Plausible provide a reference self-hosting implementation using multiple docker containers and docker-compose. This is a turnkey solution which completely abstracts away the underlying Plausible implementation.

This reference-implementation depends on an alpinelinux docker container for the Plausible binary, as well as upstream container images for Postgresql and Clickhouse.

Difficulties in the packaging process

There were two major challenges that I encountered in the packaging process.

Building Plausible Binaries for Cloudron's Ubuntu-derived base Image

Plausible is not a simple nodejs application. It is an Elixir application written in Erlang, that uses NPM for asset management. It depends on an Elixir-based toolchain that creates compiled binaries from the Plausible source code.

The Plausible team does not provide builds for their application, hence any prospective user or self-hoster that does not wish to depend upon the reference implementation must build Plausible from source.

The Plausible repository contains a Dockerfile which builds the application binary as a part of their own containerization process. My first attempt was to replicate the build process using Plausible's original Dockerfile, copy the build artifacts, and then create a Cloudron package using said builds.

Unfortunately, this did not work because Plausible's original build toolchain depends on an Alpinelinux environment. Cloudron containers use a base image that is derived from Ubuntu 22.04. Hence the build binaries for Plausible were entirely incompatible with the Cloudron docker image.

Thus for my second attempt, I setup a parallel repository that specifically created an Ubuntu 22.04-based build environment for Plausible, and used git submodules to create unofficial, Ubuntu-compatible binaries.

• https://github.com/ShenZhouHong/plausible-ubuntu-binaries

These binaries worked, and with a little additional effort, I was able to copy them over to my Cloudron app package.

Packaging Clickhouse for the Cloudron App Package

The second difficulty that I encountered was providing Clickhouse for the Cloudron app package. Clickhouse is a high-performance DBMS system often used in the analytics space, and Plausible requires it as a dependency.

Hence, any effort to package Plausible for Cloudron requires packaging Clickhouse as well.

The major difficulty that I am encountering is that Clickhouse expects to be run under the clickhouse user. This causes all sorts of obscure permissions issues when combined with gosu and the Cloudron environment. Additionally, the clickhouse database configuration is non-trivial, and so far I have yet to manage a working database connection between Plausible and Clickhouse.

Plausible Initialization

Finally, Plausible's binary must be run with a set of configuration options exposed as environment variables. These configuration options require initialization, and some of them are undocumented.

I hope these notes will be helpful to any future packaging attempts, as they took me quite a lot of time to figure out.

Plausible requires an SECRET_KEY_BASE env, which you can provision according to the documentation here. However in addition to that, it requires a TOTP_VAULT_KEY env, which is completely undocumented. The application binary will crash and segfault if it is not provided. The TOTP_VAULT_KEY env is a 32-byte base64-encoded string. An example for it can be found at Plausible's env.dev file.

Finally, on initial setup, Plausible expects certain tables to be available in the PostgreSQL database. If those tables do not exist, the binary crashes and segfaults. Plausible provides a createdb.sh script which initializes its database, but it does not work on Cloudron because it expects an empty PostgreSQL server and tries to create its own database. createdb.sh is unable to take Cloudron's existing database (which is created automatically per-application) and simply create tables within it.

A workaround is available for this issue, by running Plausible's migrate.sh instead. The script exits with a non-zero exit code, because there is nothing to migrate, but it does create the right tables. Thus as a part of the application's initialization process, the packager must be careful to run the migration script in order to initialize Plausible's PostgreSQL database.

Summary

In conclusion, Plausible Analytics is a challenging app to package for the Cloudron environment, chiefly due to three difficulties:

• A non-trivial Erlang-based build process which must create compatible binaries for Cloudron's Ubuntu-derived base image
• Dependency on Clickhouse, a high-performance DBMS which is difficult to configure, and has obscure permissions-related issues due to dependency on clickhouse user.
• Database initialization is poorly documented and requires a workaround.

Right now, I was able to successfully create a build toolchain that outputs compatible Ubuntu binaries. I was also able to solve the PostgreSQL database initialization issue with the aforementioned migrate.sh workaround, and I am even able to see the Plausible admin dashboard.

My only outstanding difficulty is with Clickhouse, as I can't seem to get it to read the config.xml files, nor write to files in the right place.

Thus, this is a call-for-help. Can somebody help me get Clickhouse working within a Cloudron application package? The most difficult part of Plausible's packaging process was the build toolchain, and I was able to get that working. If we can get Clickhouse to work, I am confident that we can release an app package for Plausible in due course.

• https://github.com/ShenZhouHong/plausible-ubuntu-binaries

[Sydney]

Update: I'm almost done packaging the app! This one was quite difficult! I've managed to make Clickhouse available in a secure, and reasonably elegant fashion, and I've gotten to the point where I can get Plausible working within my Cloudron setup.

Right now, the only task I have left is to use supervisord to manage the Clickhouse and Plausible processes within the same Docker container. Once I complete this, I will release a repository for testing.