Real hosting support needed - nginx-php-apache-postgresql-memcached-nodejs

[girish]

@b247_eu You bring up some excellent points. I think developing custom apps for Cloudron can be intimidating coming in straight from a VPS/shared hosting world. I can create some tutorial/informational post to help out. Before I do that, can you tell me what kind of platform you have used in the past for hosting your app? For example, do you have experience with something like Heroku/Dokku/Swarm and would your app easily fit in those platforms?

[b247_eu]

Dear @girish,

My full-time job is to develop web-based applications. Those applications are usually developed and then hosted in managed VPSs/metal servers. The important word here is "managed"; because all I do is to ask for a hosting configuration and then this is pushed by a sysadmin just as requested, I don't care how they do it.

So, a webserver (apache) + a php handler (let it be DSO as long as apache is running only my app, but normally php FPM should be the new default) + a postgres database with just a small extra configuration (one variable inside the postgresql.conf) + a NGINX server in front of apache (also with few custom location directives) + a memcached server, is all I need.

I already suppose that all of the aforementioned server software are updated to maintained versions, perfectly tuned for performance and security (A+ on SSL, mod_security, database config for SSD usage, etc., you name it). And of course, just if i ever need to host a damn simple WordPress presentation site or maybe opensource e-commerce, a varnish server shall be added to the stack

Are those requirements intimidating?
Shall be a customer private data be hosted on an unmanaged server or hosting solution?
Could the above plan be implemented using containers? Of course, I have used this approach in the past (inside my computer, now cloud VMs are cheaper and even free to bother with).
Could the above plan be implemented on cloudron containers? This is where you have to answer.

I won't be bothering you anymore with this topic, if the answer is yes (and by yes I suppose everything in place - for me just connection details for S/FTP/S + logs + database will suffice).

[girish]

@b247_eu I think Cloudron is not the correct platform for the development/deployment with the approach you have in mind. In your world, you have sysadmins to manage servers, you can give them requirements and they will figure out how to deploy and keep things maintained. Cloudron is more like a PaaS similar to Heroku/ECS/EKS/Google App Engine etc.

If you already know what PaaS is and how it's different from your world, you can stop reading here

In PaaS world, you as a developer, instead rely on managed services (like databases, caches, storage etc) provided by the platform for development and development (keyword is 'and', there is no sysadmin). Choosing such platforms necessarily means that you have to change and re-architecture your application to fit the platform. So, unlike the sysadmin world, where you can ask for say varnish, you have to instead look into what the platform can provide and adjust the app accordingly.

The changes you have to make can be quite extensive if it's an existing app. For example, to take some aspects of the app you mentioned: you don't generally fine tune nginx (in most cases, you don't even know what software the reverse proxy is), you don't store things in the filesystem to hand off to nodejs (you instead use an object store like s3/minio and have nodejs read from there), nodejs will be run as a separate microservice/app, static things are held in a cdn or object store, caching will be used as a service (and may not be the exact caching software you had in mind) etc. It's a very service based architecture in the PaaS world.

To conclude, the PaaS mindset is just different (whether it's better or not, it's for you to decide based on your app and the resources you have)

[b247_eu]

Hi again,
@girish , first, please accept my apologies for being ignorant or otherwise not quite ok in this relation.
You are right, my app is already developed with offloading strategy concepts (eg delegate some database load to memcached, delegate assets downloads to nginx). I think that in a serverless architecture there is no need to do that (where databases and cloud storage automatically scales based on current loads). I don't mind taking a new learning curve (regarding cloudron) if one available to provide this support.
Indeed, I can change some of my app logic to fit inside cloudron paas, and here is the state of what I need, what is provided, and what could be provided within cloudron:

• Database support: I need postgresql database and access to some postgres contrib modules (eg hstore):
  OK - cloudron provide the postgresql image/container/add-on with all (required by my app ) contrib modules;
• Database support: I need to add a postgres custom config variable to the play; usually this is done by editing postgresql.conf file and add there this var; this can't be done in cloudron postgres container directly but I think this can be done on cloudron at the database logic (eg: ALTER DATABASE db_name SET myappname.key TO 'something'; ) and if yes then everything fine regarding database; I'm pretty sure this can be done on cloudron but if one could check/confirm will be perfect.
• I need an Apache webserver and a php handler (DSO is perfect for cloudron environment):
  OK, just a quick (maybe dumb) question (and I think i have an obsession regarding this - I guess that trying to be "green" by doing/requesting optimizations is my second nature), would be possible (maybe through manifest.json) to delegate Apache .htaccess entries to vhost.conf files and then disable AllowOverride?

I need a way to efficiently deliver protected assets (eg download files that request an auth layer in front): I'm currently using nginx for that (auth by php) but this approach requires some location entries to the nginx server (app) block; Again I was thinking (because the nature of nginx that can include extra custom conf files) that this can be achieved throught manifest.json; Otherwise I have two approaches: a) let php do the entire job alone; b) use a minio server for storage; In cloudron the a) approach is OK, can be b) used (eg add a minio server to the cloudron stack)?

[mehdi]

Hi @b247_eu !

• Database support: I need postgresql database and access to some postgres contrib modules (eg hstore):
  OK - cloudron provide the postgresql image/container/add-on with all (required by my app ) contrib modules;

Good that all the contrib modules you need are there, it would have needed an update to Cloudron to add anothre one!

• Database support: I need to add a postgres custom config variable to the play; usually this is done by editing postgresql.conf file and add there this var; this can't be done in cloudron postgres container directly but I think this can be done on cloudron at the database logic (eg: ALTER DATABASE db_name SET myappname.key TO 'something'; ) and if yes then everything fine regarding database; I'm pretty sure this can be done on cloudron but if one could check/confirm will be perfect.

I am not sure that is possible. I do not think it is possible in the current model... I am quite curious as to why you need this?

• I need an Apache webserver and a php handler (DSO is perfect for cloudron environment):
  OK, just a quick (maybe dumb) question (and I think i have an obsession regarding this - I guess that trying to be "green" by doing/requesting optimizations is my second nature), would be possible (maybe through manifest.json) to delegate Apache .htaccess entries to vhost.conf files and then disable AllowOverride?

This is easy to do, no problem. You can install and control your own Apache, and do whatever you want with it. This has nothing to do with manifest.json, it can be done completely inside your app.

I need a way to efficiently deliver protected assets (eg download files that request an auth layer in front): I'm currently using nginx for that (auth by php) but this approach requires some location entries to the nginx server (app) block; Again I was thinking (because the nature of nginx that can include extra custom conf files) that this can be achieved throught manifest.json; Otherwise I have two approaches: a) let php do the entire job alone; b) use a minio server for storage; In cloudron the a) approach is OK, can be b) used (eg add a minio server to the cloudron stack)?

This cannot be done with manifest.json, there is currently no way to modify the main nginx reverse-proxy (and I think allowing it would be a bad idea).

Doing A, and letting Apache do everything, is a possibility.

About B, Cloudron does have a minio app, but it would be a separate app, on a different domain, so I do not think it's what you are looking for.

Third possibility, you could add an nginx inside your app's container, in addition to apache, or replacing it entirely.

PS: To be clear, I'm not part of the Cloudron team, just another Cloudron user who's packaged a few apps for cloudron

[girish]

@girish , first, please accept my apologies for being ignorant or otherwise not quite ok in this relation.

Oh, no need to apologize at all I am very happy that you are exploring how your app can fit into Cloudron architecture.

I need to add a postgres custom config variable to the play

Like @mehdi , I too am curious why you need this. Can you point me to some postgres docs, so I can learn more about this? Is this for stored procedures?

I need a way to efficiently deliver protected assets

I think you already figured the correct approach. You can store the assets in minio (which is an app on Cloudron). You can then use minio keys to read/write from your app. I would decide to use (or not use) minio based on how big these assets are. If these are just small files, you can just let php handle it. If it's video files, streaming etc minio would probably do a better job (and will also help you that you can put this in a CDN later which many services like DO Spaces/AWS S3 support. It will be easy to migrate since minio uses S3 API).

[b247_eu]

The postgres custom configuration parameter is used in my app logic for database tables row-level audit purposes. As in git where we have diffs and rollback strategies for files but this time for database entries. And here: https://stackoverflow.com/questions/34476062/setting-a-configuration-parameter-for-functions-implemented-in-pl-pgsql is the perfect question and answer that reflects my needs, including the accepted answer, were found there is a way to bypass server global config and rely on defining custom parameters for a database (the "ALTER database ... SET custom.param ...." SQL command approach).

Regarding static assets delivery, there are app (my app) users (logged in members) uploaded files (not for public download), and the upload size and types (mainly videos, docs, and archives) have only a few limits. And nginx does a perfect job when handling this kind of files, especially for concurrent hits. So this would be my first approach because the app is already filled up with a lot of uploaded files. With a config in my php-app I can delegate the download hard work to php (this is already codded and optimized). The third option (minio) is a great approach but not (yet) implemented as a storage backend in my app. So I guess I will let minio for further app upgrades and rely for the moment only on local storage. p.s. could minio be added/run as an "addon" (like mysql, redis, localstorage, postgres)?

I still have one more question before spawning a playground VPS (with cloudron PaaS inside) to try to build/run/install a custom cloudron app for the need.
I've advanced in cloudron learning curve and here:
https://docs.cloudron.io/custom-apps/guide/ found this:
"The nginx configuration, provided with the base image, can be used by adding an application specific config file under /etc/nginx/sites-enabled/ when building the docker image." along with "ADD <app config file> /etc/nginx/sites-enabled/<app config file>" dockerfile line and a app-nginx.conf example (port 8000).
Is above documentation still up to date, does this means that I can add my custom nginx conf file to the image?

[mehdi]

@b247_eu said in Real hosting support needed - nginx-php-apache-postgresql-memcached-nodejs:

Is above documentation still up to date, does this means that I can add my custom nginx conf file to the image?

Yes of course, you can run an nginx inside your container and apply whatever configuration you want to it.

To be clear, it would apply only to the nginx inside your app, which itself would still be behind the main nginx reverse-proxy of the cloudron paltform.

[b247_eu]

In documentation it says: "The nginx configuration, provided with the base image, can be used by adding an application specific config".
In my understanding it seems that all i have to do to add my custom config is:

FROM cloudron/base:2.0.0

CLOUDRON related stuff eg WORKDIR

ADD nginx-custom.conf /etc/nginx/sites-enabled/nginx-custom.conf

OTHER CUSTOM STUFF required by my app

Am I wrong? It will work like that? Will cloudron forward ports to my container port 8000 if

server {
  listen 8000;

  root /app/code/dist;

  location /api/v1/ {
    proxy_pass http://127.0.0.1:8001;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 86400;
  }
}

will be my custom configuration?

@mehdi, you've say that you "packaged" few apps for cloudron, can you tell us one authored by you? like this one authored by @girish : https://git.cloudron.io/cloudron/wordpress-unmanaged-app/-/tree/master

[mehdi]

@b247_eu said in Real hosting support needed - nginx-php-apache-postgresql-memcached-nodejs:

In documentation it says: "The nginx configuration, provided with the base image, can be used by adding an application specific config".

Yeah, nginx ins provided with the base image in the sense that the nginx binary is pre-installed.

@b247_eu said in Real hosting support needed - nginx-php-apache-postgresql-memcached-nodejs:

In my understanding it seems that all i have to do to add my custom config is:
FROM cloudron/base:2.0.0

CLOUDRON related stuff eg WORKDIR

ADD nginx-custom.conf /etc/nginx/sites-enabled/nginx-custom.conf

OTHER CUSTOM STUFF required by my app

This would be enough to change the config for said nginx, yeah, but you still have to run it in your start.sh !

@b247_eu said in Real hosting support needed - nginx-php-apache-postgresql-memcached-nodejs:

@mehdi, you've say that you "packaged" few apps for cloudron, can you tell us one authored by you? like this one authored by @girish : https://git.cloudron.io/cloudron/wordpress-unmanaged-app/-/tree/master

https://git.cloudron.io/cloudron/openvpn-app was originally done by me, but now it's maintained by the cloudron team

https://git.cloudron.io/mehdi/river/ this one could be interesting for you, as I run multiple services in the same container, including an nginx

https://git.cloudron.io/mehdi/orthanc is also done by me and awaiting review to be on the official store