Prometheus - monitoring & time series database

[marcusquinn]

And Thanos:

"Open source, highly available Prometheus setup with long term storage capabilities."

https://thanos.io
https://github.com/thanos-io/thanos

[marcusquinn]

And AlertManager:

"The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts."

https://prometheus.io/docs/alerting/latest/alertmanager/
https://github.com/prometheus/alertmanager

[rossmaclean]

After installing Grafana on Cloudron, I'd really like this too!

[eriktad]

https://git.cloudron.io/erik/prometheus-app our app for Prometheus ( in progress )

[girish]

@eriktad This looks like a good start! I guess we need some sort of authentication for the prometheus dashboard? https://prometheus.io/docs/guides/tls-encryption/ has some information.

[eriktad]

@girish Yes but I only find basic http auth on Prometheus docs, so what can you suggest?

[eriktad]

@girish is it normal practice for us?

[girish]

@eriktad Yes. You can do something like https://git.cloudron.io/cloudron/simple-torrent-app/-/blob/master/apache/cloud-torrent.conf . That apache config sets up the app for authentication via Cloudron LDAP .

[eriktad]

@girish cool thx

[eriktad]

@girish I tried this example, but somehow apache does not start for me, if you can help, I will give the more information about this

[girish]

@eriktad Happy to help, let me know what error you are facing.

[eriktad]

@girish if I try to run start.sh manually it gives me

[eriktad]

@girish So idk what happens I try to do it like another example I found in GitLab

[eriktad]

@girish Ye problem founded. #!/bin/sh -> #!/bin/bash

[eriktad]

@girish Prometheus app is done. Can you review it and tell me what to need to do next?

[eriktad]

@girish https://git.cloudron.io/erik/alertmanager-app and Alertmanager app

[girish]

@eriktad That's great progress! Yes, I can take a look next week. I have marked this app as WIP.

[girish]

@eriktad said in Prometheus - monitoring & time series database:

@girish https://git.cloudron.io/erik/alertmanager-app and Alertmanager app

I don't have much experience with Promotheus. Does it make sense to merge alertmanager into the main app itself or do you think it should be separate apps?

If you need help deciding, we can take a guess from how they are usually deployed? Are they in separate domains or as subpaths etc.

[eriktad]

@girish Yeh, I thought about that but idk, they run on other ports and have other dashboards, but altermanager only alerting Prometheus massages, I can try to merge, think it is good idea, but we need to understand if user no need alerts why we setup it with Prometheus.

[marcusquinn]

@eriktad They make sense as separate Apps.

I'd suggest the App naming would make it clearer, so:

• Prometheus Server
• Prometheus Alertmanager
• Thanos (Single-Node)

For interest, like Statping, in mission-critical setups it makes sense to have Prometheus running on a separate Cloudron instance for continuity of data-gathering throughout the ups & downs of the services being monitored.