@agitk this is the Cloudron forum . Here we discsus the Promotheus package of Cloudron . Are you using Cloudron for hosting Prometheus?

But regardless, app questions are probably best asked in https://github.com/prometheus/prometheus/discussions (I think).

please post your yaml with removed / redacted sensitive information in a code block and not plain.
You can also always use

e.g. :

job_name: <SYSTEM-NAMEN> scrape_interval: 5s static_configs: - targets: ['<MY-IP:9100']

yamllint is also a great to to check if you yaml file is valid.

@girish Hi Girish! So I could totally be misreading this, but after some testing, it really does look like CPU and Mem stats do come through from the underlying host... Here is what I tried to confirm this. Let me know if I am way off base and I'm missing something crucial.

First, just base observations:

Prometheus reports back 4GB of total system memory, much more than the 512MB I have allocated for the Prometheus app container itself. Most of the time, the CPU utilization of my host VPS hovers around 2-3% as reported by Prometheus data, whereas the (according to Top) CPU utilization of the container is 1-2.5% Graphs on my VPS provider's control panel roughly correlate to the CPU utilization observed in Prometheus data.

Second, based on stress-testing

Using Stress to stress-test individual CPUs or all CPUs to 100% in the host system correctly registers corresponding observations in Prometheus data. Using Stress -vm within the Prometheus app container to take up 256MB of RAM correctly registers a ~256MB bump over existing baseline memory observations from Prometheus data, and not 50+% utilization as would be expected if the container stats were being reported (container has 512MB allocated to it). Using Stress -vm out on the host system to chew up 2 GB of memory correctly shows a ~2GB increase in memory utilization reported by Prometheus data.

So far CPU, Mem, and Network stats are all that I have played with, and Network is the only one so far that I can confirm positively does only report the container stats instead of the host system.

Not having correct network stats isn't a deal-breaker for me (though it would be nice, but not if doing so would put the system at risk of a container breach). I thought I'd at least report my findings thus far for anyone else that stumbles upon this.

Anyway, even if I am wrong, I have had a real great time playing with this! Thanks for making it all possible!

@girish Pefect. As always, thank you for the quick response!

@lcd_official indeed, prometheus_variables.sh is obsolete by now. Use env.sh instead.

I have adjusted the latest package to make CLI args customizable (since there's no way to do this via env vars).

@JLX89 prometheus app already supports proxyAuth . Can you check with a new installation ? We even have basic auth turned on.

@Jan-Macenka yeah, there's some bug in the fs module we use . We are replacing it next release, so maybe this bug goes away.

Blocking /metrics and /health endpoints for Apps by default

I'd also like to track Jellyfin which I love to run as an cloudron App. It comes with pre installed Prometheus endpoint according to the documentation which I'd like to use. The default Cloudron-location for this file is /app/data/jellyfin/config/system.xml
However I dont want to expose this endpoint to the public Internet. Any Ideas how I could prevent this?

If we find a good solution, I think it would be best to have the /metrics and /health endpoints blocked by default because Cloudron-Admins most likely dont want to expose these to the Internet anyway.

@girish would you have an Idea where I could apply such a config change?

Best regards and thanks for the answer in advance 🙂

PS: Will gladly contribute to the Documentation as we found a good solution. I just fell down the rabbit hole to a certain degree in advancing my home-lab more towards a real data-center 🙂

Thanks @girish and sorry for the late reply

@cortex said in Adding local targets to Prometheus:

How do I add the other Cloudron apps to the targets to monitor them?

I don't have much idea here, maybe @eriktad or @BrutalBirdie knows. In general, what is needed to add a target to Prometheus? IP Address?

@brutalbirdie thanks, I merged it and made a new package release.

@fbartels it works, thanks!

@christophermag This should be fixed now.