Centralized logging?

[malvim]

Hey, guys.

Just to check if this is something that's planned for cloudron sometime in the future?

I suppose I could package something like logstash, but not sure if we'd be able to just capture whatever the containers are logging to stdout/stderr, without having to dive into box code... What do you think?

I think it would be neat to have some kind of centralized logging (as an app, maybe?) instead of clicking each app through the admin UI, or running cloudron logs for each one.

What do other people think about this?

Thanks!

[marcusquinn]

Agreed, although practically LogStash is our ideal hope:

• https://forum.cloudron.io/topic/2811/logstash

[nebulon]

Cloudron already basically manages logs, so I assume you are asking mostly for improvements in the log viewer. Currently we have placed log viewer buttons in context within the dashboard, but maybe those are not as convenient and well placed as they should be. Do you have specific ideas what could make life easier with the logs? The log viewer as such hasn't changed since some time now, so maybe by now it needs some more features and adjustments.

[marcusquinn]

Not sure you'd want to re-invent all this though: https://www.elastic.co/logstash

We use it to monitor logs and alert relevant errors in relevant Discord channels to be pro-active on error handling.

[malvim]

@nebulon I like the log viewer as it is, and I'm not talking about changing or improving it in any way (though it might need improvements, it's just not what I'm currently talking about heh).

What I mean is centralized logging for all apps, like LogStash and others do. So you can see all logs in the same place, get a feel for the overall amount of errors, filter messages by type, get some insights on the overall health of your apps, so on.

It's probably easy enough to package something and deploy it to a cloudron, I'm just not sure we would be able to collect log data as inputs to the app. Is there a log viewer API, or something like that? Does the log viewer itself rely on an API to get the messages? Maybe that could be (ab)used to use as input to the central logging app...

[girish]

I think that's a good idea in general. A log view that shows unified logs across the full platform. Whether that's best achieved by pushing all logs to some external party or by implementing something of our own, we have to investigate. Overall requiring any external software means putting the burden on the user to install, maintain and manage it.

[malvim]

@girish Yeah, that makes total sense. If it's something you guys think you should provide, then I completely agree with you. The upside of having this is an app, though, is having the user choose whether they want this or not. I agree it might be easier for you to just implement something based on the log viewer you already have...

Either way, I got curious about the API, is there a log API where your log viewer gets its data from?

[girish]

@malvim Yes, you can use the /api/v1/apps/<appid>/logs and /api/v1/apps/<appid>/logstream API. I see that it's missing from our docs but you can see the CLI code to see how it's done - https://git.cloudron.io/cloudron/cloudron-cli/-/blob/master/src/actions.js#L895

[nebulon]

I've put an initial documentation for both APIs at https://cloudron.io/documentation/api/#tag/Apps/paths/~1apps~1{appid}~1logs/get

[girish]

To add to what @nebulon said, https://cloudron.io/documentation/api/#tag/Services/paths/~1services~1{service}~1logs/get is for the addon logs and https://cloudron.io/documentation/api/#tag/Tasks/paths/~1tasks~1{taskId}~1logs/get for task logs.