Any issues with including NetData on the root server and as an app add-on?

[cvachery]

Hello,

I'm currently looking at this topic at the moment and maybe it would be possible to split this issue in two different subject.

I understood that the main issue to add netdata to cloudron as an app is the capabilities required from the docker image to be able to collect all the metrics of the host. Which I understand is a problem as it challenge the whole security design actually implemented.

But would it be conceivable to deploy netdata as a cloudron app with limited monitoring capabilities as a known limitation. I know that it can seems counter-productive but I have a specific purpose for which it could be useful : Using this netdata instance as a parent node to centralize all the metrics from different children and use the ldap/proxyauth addon of cloudron to add authentication to the WebUI
That would be a really great use case for me.

And concerning the other subject of actually collecting the data of the cloudron host using netdata. IMHO a tutorial on how to deploy it with docker and /or docker-compose, and the firewall configuration needed would be enough for most admin I think.

[marcusquinn]

@cvachery I believe it makes most sense to install in the root host.

I assume being installed there would give it a view of all Docker Containers, anyway.

Perhaps it should just be an optional install on the root host, with disclaimers that issues might not be supported.

[cvachery]

@marcusquinn I do agree that for a monitoring point of view it indeed makes more sense to deploy on the root host (that was my second point) but as a cloudron app I do believe the not privileged app has an utility.

And using this architecture of a netdata parent node as a cloudron app and a child node being deployed on the root host streaming its data to the parent would be then transparent and we would have the best of the two world if the installation on the root host is at least well documented on the cloudron side or even as you suggest made an optionnal part of the install process

[robi]

@cvachery Start a package for it using the docs and see how far you get, others here can chime in to help. Bit by bit, progress is made.

[cvachery]

I've had some time this week to work on this topic. So here is where I am at the moment.
I packaged an app for cloudron, it does seem to be working properly. (I let you guys have a look/test as it is my first try at packaging for cloudron)

The only thing I can't get to work is the streaming part. (eg a child node stream to my cloudron node which is a parent), my best guess is that the reverse proxy used by cloudron doesn't send the tcp stream to the container but only http. But I don't know how to configure/fix that.

The netdata configuration file is in /app/data/etc so to set up streaming you can add a file stream.conf using the file explorer and restart the app

Infos I can give :
Command used to install : cloudron install --image cvachery/netdata-app:v1.44.1
Github : https://github.com/aylham/cloudron-netdata
Docker image : docker pull cvachery/netdata-app:v1.44.1
Netdata doc on how to setup streaming : https://learn.netdata.cloud/docs/streaming/understanding-how-streaming-works#enable-streaming-between-nodes

[girish]

@cvachery said in Any issues with including NetData on the root server and as an app add-on?:

The only thing I can't get to work is the streaming part. (eg a child node stream to my cloudron node which is a parent), my best guess is that the reverse proxy used by cloudron doesn't send the tcp stream to the container but only http. But I don't know how to configure/fix that.

Not sure if this helps but if the data is sent in separate tcp port, you have to define it using tcpPort in manifest - https://docs.cloudron.io/packaging/manifest/#tcpports

[cvachery]

Thanks for the reply @girish
I don't know how I missed this option.
It does work now when the TCP port is defined (it is the same as httpPort, but it does have to be explicitly enabled)

Anyway it is now working as expected, their even is the host metrics on the overview.
Still have some quality of life configuration changes to do but the app is functional

[girish]

@cvachery very nice! can you put a LICENSE in your repo as well? We will need some open source license (MIT, BSD etc) for us to use your package code. I will add this to our internal list of app to review.

[cvachery]

@girish Licence file added

[girish]

@cvachery thanks, will take a look next week or so!

[cvachery]

Hello @girish,
Any chance you had time to have a look a this?

[girish]

@cvachery not yet, but it's on our list.

[Kubernetes]

@cvachery said in Any issues with including NetData on the root server and as an app add-on?:

Infos I can give :
Command used to install : cloudron install --image cvachery/netdata-app:v1.44.1
Github : https://github.com/aylham/cloudron-netdata
Docker image : docker pull cvachery/netdata-app:v1.44.1
Netdata doc on how to setup streaming : https://learn.netdata.cloud/docs/streaming/understanding-how-streaming-works#enable-streaming-between-nodes

Do you provide the CloudronManifest.json somwhere?

[girish]

@Kubernetes it's in the repo at https://github.com/aylham/cloudron-netdata

[cvachery]

And hopefully it will do the job.
I use my package as a parent node since I released it without any issue but more usage will be helpful

[d1rk]

Sounds promising. Can someone please share how to install this app on my Cloudron? Thank you.

[Lanhild]

@d1rk This application package not being officially supported by the team, it means you'd need to follow the instructions from that package repo. However, you could use https://docs.cloudron.io/packaging/tutorial/ as an example of what installing a custom application looks like.

[cvachery]

After to be honest I'm encountering an issue that is the backups aren't working for some reason.
Does someone who installed my package have this issue too. Or is it just from my install?

[girish]

@cvachery this might be a general Cloudron backup issue. Does netdata create a lot of temporary files? If possible, change the directory where it changes those files to be outside the backup (like in /run or /tmp). A fix from the Cloudron side is coming (it's a bit complicated since we have to rewrite/replace a node module).

[cvachery]

After update and reboot of the Cloudron backup seems to be working OK now. So probably the bug you said on Cloudron side.

What would be the next steps to integrate it directly into Cloudron AppStore?