AppsMonitor - monitoring for your Cloudron

[timconsidine]

AppsMonitor is now available as a Community App

AppsMonitor is a lightweight monitoring dashboard for Cloudron that gives a quick view of app health, services, disk usage, recent events, and alerts.

Current features include:

• dashboard for app and platform status
• issue detection and recent history
• email and Telegram alerts (instant for issues, daily digest as summary)
• public status page (optional)
• Cloudron proxyAuth protection

It is packaged as a Community App and built on the new cloudron-minibase.

CloudronVersions.json : https://communityapps.appx.uk/cloudron-appsmonitor/CloudronVersions.json
(also on https://ca.cloudron.io)

A nice bonus from using cloudron-minibase: the current app image is 307MB.

If anyone gives it a try, feedback on usability, alerting, and anything that looks off would be very welcome.

[robi]

This should be part of Cloudron default features.

[necrevistonnezr]

@timconsidine
Did that install just kill the nginx.service?
Since installing that, Cloudron doesn’t come up.

The log I could see before Cloudron went down:

Sep 16 07:27:06 apptask: run: app error for state pending_install: BoxError: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null
    at file:///home/yellowtent/box/src/reverseproxy.js:103:26
    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
    at async retry (file:///home/yellowtent/box/src/retry.js:13:20)
    at async reload (file:///home/yellowtent/box/src/reverseproxy.js:101:5)
    at async writeAppConfigs (file:///home/yellowtent/box/src/reverseproxy.js:572:5)
    at async Object.configureApp (file:///home/yellowtent/box/src/reverseproxy.js:594:5)
    at async installCommand (file:///home/yellowtent/box/src/apptask.js:504:5) {
  [stack]: [Getter/Setter],
  [message]: 'Error reloading nginx: /usr/bin/sudo exited with code 1 signal null',
  name: 'BoxError',
  reason: 'Nginx Error',
  extra: {}
}
Sep 16 07:27:06 reverseproxy: Attempt 3 failed. Will retry: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null BoxError: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null
    at file:///home/yellowtent/box/src/reverseproxy.js:103:26
    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
    at async retry (file:///home/yellowtent/box/src/retry.js:13:20)
    at async reload (file:///home/yellowtent/box/src/reverseproxy.js:101:5)
    at async writeAppConfigs (file:///home/yellowtent/box/src/reverseproxy.js:572:5)
    at async Object.configureApp (file:///home/yellowtent/box/src/reverseproxy.js:594:5)
    at async installCommand (file:///home/yellowtent/box/src/apptask.js:504:5)
Sep 16 07:27:06 tasks: setCompleted - 24688: {"result":null,"error":{"message":"Error reloading nginx: /usr/bin/sudo exited with code 1 signal null","reason":"Nginx Error","stack":"BoxError: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null\n    at file:///home/yellowtent/box/src/reverseproxy.js:103:26\n    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)\n    at async retry (file:///home/yellowtent/box/src/retry.js:13:20)\n    at async reload (file:///home/yellowtent/box/src/reverseproxy.js:101:5)\n    at async writeAppConfigs (file:///home/yellowtent/box/src/reverseproxy.js:572:5)\n    at async Object.configureApp (file:///home/yellowtent/box/src/reverseproxy.js:594:5)\n    at async installCommand (file:///home/yellowtent/box/src/apptask.js:504:5)"},"percent":100}
Sep 16 07:27:06 tasks: updating task 24688 with: {"completed":true,"result":null,"error":{"message":"Error reloading nginx: /usr/bin/sudo exited with code 1 signal null","reason":"Nginx Error","stack":"BoxError: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null\n    at file:///home/yellowtent/box/src/reverseproxy.js:103:26\n    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)\n    at async retry (file:///home/yellowtent/box/src/retry.js:13:20)\n    at async reload (file:///home/yellowtent/box/src/reverseproxy.js:101:5)\n    at async writeAppConfigs (file:///home/yellowtent/box/src/reverseproxy.js:572:5)\n    at async Object.configureApp (file:///home/yellowtent/box/src/reverseproxy.js:594:5)\n    at async installCommand (file:///home/yellowtent/box/src/apptask.js:504:5)"},"percent":100}
Sep 16 07:27:06 BoxError: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null
    at file:///home/yellowtent/box/src/reverseproxy.js:103:26
    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
    at async retry (file:///home/yellowtent/box/src/retry.js:13:20)
    at async reload (file:///home/yellowtent/box/src/reverseproxy.js:101:5)
    at async writeAppConfigs (file:///home/yellowtent/box/src/reverseproxy.js:572:5)
    at async Object.configureApp (file:///home/yellowtent/box/src/reverseproxy.js:594:5)
    at async installCommand (file:///home/yellowtent/box/src/apptask.js:504:5)
Sep 16 07:27:06 Exiting with code 0
Sep 16 07:27:06 taskworker: Task took 39.389 seconds
Sep 16 07:26:40 WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Sep 16 07:26:40 Press CTRL+C to quit
Sep 16 07:26:40 * Debug mode: off
Sep 16 07:26:40 * Running on all addresses (0.0.0.0)
Sep 16 07:26:40 * Running on http://127.0.0.1:3000

Troubleshoot info:

═══ Host ═══
Vendor: AZW Product: EQ
Linux: 7.0.0-31-generic
Ubuntu: resolute 26.04
Cloudron: 10.0.5
Execution environment: none
none
Processor: Intel(R) N100 x 4
RAM: 15616096KB
Disk: /dev/mapper/ubuntu--vg-ubuntu--lv  193G

═══ System ═══
[OK]Root disk usage is OK (57%)
[OK]Memory usage is OK (44%)
[OK]Disk I/O on /home/yellowtent (ubuntu--vg-ubuntu--lv, unknown): 461 MB/s write, iowait 1%
[OK]Clock is NTP-synchronized
[OK]node version is correct
[OK]IPv6 is enabled in kernel. Public IPv6 address detected
[OK]docker is running
[OK]docker version is correct

═══ Networking ═══
[OK]no incompatible persistent-firewall packages installed
[OK]DNS is resolving via systemd-resolved
[OK]unbound is running
[OK]IPv4 HTTPS to api.cloudron.io/api/v1/helper/public_ip
[OK]IPv6 HTTPS to api.cloudron.io/api/v1/helper/public_ip
[OK]IPv4 HTTPS to auth.docker.io/token
[OK]IPv6 HTTPS to auth.docker.io/token
[OK]IPv4 HTTPS to acme-v02.api.letsencrypt.org
[OK]IPv6 HTTPS to acme-v02.api.letsencrypt.org
[OK]netplan is good

═══ Cloudron Platform ═══
[OK]MySQL is running
[FAIL]nginx is down. Removing extraneous dashboard domain configs ...
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.
[FAIL]nginx is still down, please investigate the error by inspecting 'journalctl -u nginx' and /var/log/nginx/error.log
[OK]Dashboard cert is valid (83 days remaining, expires Dec  8 14:12:24 2026 GMT)
[FAIL]Could not load dashboard website with loopback check
[OK]No pending database migrations
[OK]Service 'mysql' is running and healthy
[OK]Service 'postgresql' is running and healthy
[OK]Service 'mongodb' is running and healthy
[OK]Service 'mail' is running and healthy
[OK]Service 'graphite' is running and healthy
[OK]Service 'sftp' is running and healthy
[OK]box v10.0.5 is running
[FAIL]IPv4: could not load https://my.domain.net
Hairpin NAT may not be working. Please check if your router supports it
[FAIL]IPv6: could not load https://my.domain.net
[OK]Domain domain.net is valid (170 days remaining, expires 2027-03-05T10:29:03Z)

======== Summary ========
PASS:  29
WARN:  0
FAIL:  5
SKIP:  0

Failures:
  - nginx is down. Removing extraneous dashboard domain configs ...
  - nginx is still down, please investigate the error by inspecting 'journalctl -u nginx' and /var/log/nginx/error.log
  - Could not load dashboard website with loopback check
  - IPv4: could not load https://my.domain.net
  - IPv6: could not load https://my.domain.net

[necrevistonnezr]

This seems to be the culprit, “mon.domain.net” was the domain I tried to use:

[emerg] directive "location" has no opening "{" in
/etc/nginx/applications/ad146d5a-3e5f-417c-a377-7bc89fad201a/mon.domain.net.conf:181

So for now: DO NOT INSTALL

[necrevistonnezr]

@girish @timconsidine
Or maybe it’s a bug with HTML-escaped quotes in proxyAuth nginx config?

nginx fails to start:

[emerg] directive "location" has no opening "{" in /etc/nginx/applications/<app-id>/mon.domain.net.conf:181

The app seems to use the proxyAuth addon. The generated location block contains HTML entities instead of literal single quotes:

Had to use a screenshot, as the HTML entity ' gets converted on a block quote on this forum

nginx treats "#" as the start of a comment, so the opening brace is lost and the config test fails, which takes down nginx for all apps.

Environment: Cloudron 10.0.5, Ubuntu 26.04, kernel 7.0.0-31-generic

[girish]

Oh, that looks like a bad validation bug. Testing. Looks like I broke this when I had "hardened" the nginx proxy auth. Previously, nginx config injection was possible from malicious community apps (by putting a payload in proxy auth paths).

[girish]

@timconsidine https://communityapps.appx.uk/cloudron-appsmonitor/screenshot.png is a 404

[girish]

Thanks, so the proxy auth addon should be fixed - https://git.cloudron.io/platform/box/-/commit/6705b1c1107ec43ce09eff1945cd84ef9c60ff07

[timconsidine]

@girish sorry, bad link, fixing now.

Correct link :
https://communityapps.appx.uk/cloudron-appsmonitor/media/screenshot.png

[necrevistonnezr]

Thanks!

Just out of curiosity (and little technical knowledge): Shouldn't a system that isolates apps via docker not be taken down completely by a malformed configuration file of one app?

[timconsidine]

@necrevistonnezr thanks for reporting.

If I am understand this correctly, it wasn't me or the app

Is it working for you now ?
Not seeing any issues on a fresh install

[necrevistonnezr]

No, I de-installed as soon as the system was up and running again, I had my share of experiments for the day

[timconsidine]

@necrevistonnezr understand, sounds fraught

just done a 3rd test install
both cloudron box services and app seem ok

[timconsidine]

Just out of curiosity (and little technical knowledge): Shouldn't a system that isolates apps via docker not be taken down completely by a malformed configuration file of one app?

someone will correct me if I am seeing this wrongly.
it wasn't malformed configuration of an app
it was a box service that failed

Ideally and mostly running containers are isolated.
But every cloudron app depends on box services, from nginx to maybe addons like proxyAuth. If the box service trips up, the app can't function as it should.

[p44]

@robi Totally agree...

[necrevistonnezr]

@timconsidine Yes, this was more directed at the Cloudron Team...

[girish]

@necrevistonnezr said:

Just out of curiosity (and little technical knowledge): Shouldn't a system that isolates apps via docker not be taken down completely by a malformed configuration file of one app?

Yeah, absolutely. This is a good conversation to have

Cloudron was written primarily for pre-packaged apps written by us and not as a sandboxing environment to run arbitrary applications. When packages were written by us, we could "trust" the input in the manifest file, description, post install etc. When we made custom apps possible, the worst case was that a user sabotages himself with a poorly written manifest. But now with community apps, there is a new vector that malicious apps can do unknown things. For example, what if the description has some clickjacking or scam link? So, you can hit problems even without installing or running an app (so we don't hit run time isolation). This is also why we don't want to integrate community listing straight into the main dashboard, this would give too many good feelings of trust.

We are fixing issues (like this) as and when they come, but one should only run 3rd party packages they trust. Cloudron company's primary focus is still to runs the apps from the main app store and not to make a general trusted environment to run random stuff (FWIW, I don't think such a product exists out there).

[timconsidine]

Sounds like “CommunityAppChecker” "AppInspector" utility is needed. Put a CloudronVersions link in input box, utility checks whatever is needed/possible, generate alerts for sus content.
Conceptually best to come from Cloudron team but that drags Cloudron towards approving apps, which is probably a v.bad direction. Maybe I will give some thought to it.

I really understand and support the official stance - it is simply not viable otherwise.

Some other simple defences :

• have registered app packager scheme : no fees, basic metadata, probably already publicly known, enough to know legit packager and no more
• ca.cloudron.io only takes submissions from registered packagers (currently must have a Cloudron account)
• Cloudron will hate (understandably) any form of stars or likes on ca.cloudron.io, but maybe a simple more neutral voluntary “I installed this” counter

[timconsidine]

Could not resist it - AppInspector available :
https://forum.cloudron.io/post/129539

Needs more and deeper tests, and nothing will ever be a guarantee.
But it's a start.

[nichu42]

@timconsidine Fresh install fails with
An error occurred during the install operation: Nginx Error: Error reloading nginx: /usr/bin/sudo exited with code 1 signal null