What's coming in 6.0 (take 2)
-
@nebulon said in What's coming in 6.0 (take 2):
The volumes are setup outside of the apps first and are essentially independent. Once you've created a volume you can then mount that into an app via the Storage section in the app configuration. Even if mounted though, it will not be part of the app backup, since the main reason for volumes is to be mounted into multiple apps. For that we have to first figure out how to manage those with regards to backups.
No, I mean external volumes that you can currently already move the data dir to under „Resources“ in the app settings - this gets backed up, correct?
(
-
We haven't decided on the naming yet, but maybe Volumes can be instead renamed to External Media or something, if it clarifies things. Open to other suggestions as well.
-
@MooCloud_Matt this is only true if you use dumb S3 connectors that treat it like a filesystem instead of an object store.
It's a very different experience when things are treated properly as objects.
-
Re: Volumes - I think this will create a solution for something that I have been trying to resolve - i.e. Nextcloud app backups WITHOUT the data!
I want to be able to back up the app via Cloudron natively - but backup the data via borg/restic/kopia or the like. This looks like it could resolve this issue if the data was an attached volume. Very much looking forward to this feature if it's like that.
-
@iqweb said in What's coming in 6.0 (take 2):
Re: Volumes - I think this will create a solution for something that I have been trying to resolve - i.e. Nextcloud app backups WITHOUT the data!
Yes, I think this will solve that use case. You mount a volume into nextcloud and configure nextcloud to use it as external storage.
-
@girish can you also mount an ‘on server’ folder like that?
-
@imc67 Yes. Currently, the main restriction is that you can only add host paths under /mnt and /media as a volume. So, you can create some NFS mount or SSHFS mount there and then add it as a volume and then mount it into an app.
-
@ruihildt There is a free tier, and if the "5 Mailboxes" means there is some functional email, then that is not bad.
A life lived in fear is a life half-lived
-
@scooke I'm not sure what you're replying to?
-
@girish can't wait to have volumes to play with!
what's your guestimate ETA? -
@jdaviescoates Cloudron 6 is ~2-3 weeks away. The translation project is quite a massive change, so maybe I am being overlay optimistic here.
-
@girish all power to your elbow! (i.e. good luck!)
-
@ruihildt Your comment from 26 days ago, "As an aside, they are advertising Cloudron as free software, which is not correct at the moment." I came across it as I read through the entire thread.
A life lived in fear is a life half-lived
-
@scooke said in What's coming in 6.0 (take 2):
@ruihildt Your comment from 26 days ago, "As an aside, they are advertising Cloudron as free software, which is not correct at the moment." I came across it as I read through the entire thread.
I think you've missed the point
@ruihildt was talking about free as in freedom (which is how that project advertises Cloudron), not free as in price.
-
@jdaviescoates I figured so. But I wonder why @ruihildt bothered to point out what he did. The few lines I found on the website aren't actually advertising Cloudron as %100 open source, but rather that the code they use is open source. I'd be more bothered by their approach, like Framasoft, where if the user has no idea about the existence of Cloudron, in this case, the company's description of what they do (https://osinum.fr/) makes it sound like it is and has been all their own doing. Much of what they promise to do is based on whether Cloudron, or all the other myriad open source projects in use, has or will implement it, but they don't clearly specify that. If I found them, and started using them, and THEN discovered they actually use Cloudron, I'd be a little miffed, and would just switch over to Cloudron.
Anyway, I have no clue what "being sponsored by Medias-Cite" means, but if the Cloudron team are fine with it, and this "sponsoring" helps move Cloudron forward, great.(A bit more reading shows that they, osinum.fr, are charging a minimum 150 EURO/month!!! I guess they are paying Cloudron for the Premium Plus plan. They offer, for almost triple the price of Cloudron's pricing,
1 domain name (.fr, .com, .org, .net)
Accommodation in France
Shared server
100 GB
5 applications
50 users
Management of user groups
Daily backups
Surveillance / Monitoring / Security
Email supportMaybe I should get into offering cloudron!!
A life lived in fear is a life half-lived
-
hey guys, let's stick to Cloudron 6 related discussion here
Feel free to open new thread for other topics. -
@girish fair enough
Just to quickly say (sorry) to @scooke re:
the user has no idea about the existence of Cloudron
I'm not sure where you are looking but on https://osinum.fr/ there is this:
Which translates as:
"The core of OSINUM is the Cloudron software and its team of developers."
So 1) they seem quite clear about being based on Cloudron, and 2) they've actually changed what it used to say which @ruihildt was commenting on.
Previously it said: "The core of OSINUM is the open source software Cloudron and its team of developers."
Edit: see my screenshot here https://twitter.com/jdaviescoates/status/1314243996441620480
/end.
-
@jdaviescoates @girish D'accord.
-
@girish said in What's coming in 6.0 (take 2):
You add a host path as a volume and give it a name. This is most likely some EBS/external hard disk/Block Storage.
I'm assuming volumes will support CIFS unlike current external data storage, right? (if not I had better cancel the Hetzner storage box I just ordered for this purpuse (or just start using it for backups where CIFS is already allowed)!
) -
@girish said in What's coming in 6.0 (take 2):
@imc67 Yes. Currently, the main restriction is that you can only add host paths under /mnt and /media as a volume. So, you can create some NFS mount or SSHFS mount there and then add it as a volume and then mount it into an app.
I really would appreciate it if you could create a step by step instruction in the docs by that time. I guess more users with large base “disk” would want to use this!
-
@jdaviescoates said in What's coming in 6.0 (take 2):
I'm assuming volumes will support CIFS unlike current external data storage, right? (if not I had better cancel the Hetzner storage box I just ordered for this purpuse (or just start using it for backups where CIFS is already allowed)! )
The underlying problem is still there, CIFS / SMB don't support users and groups in the FS, so i guess it will depend on the apps, if they rely on FS permissions or not
-
@mehdi hmz, thanks, so I'm guessing likely not then.
So Hetzner storage boxes likely a no no for volumes (I guess I'll cancel the one I've just ordered, or just use it for backups)
I wonder what would be a good option...
I guess maybe Hetzner Volumes would work well, but they cost a lot more (a 500GB storage box is 5.68€/mo, whereas a 500GB Volume is 24€/mo)
-
@jdaviescoates said in What's coming in 6.0 (take 2):
So Hetzner storage boxes likely a no no for volumes (I guess I'll cancel the one I've just ordered, or just use it for backups)
So, there's two kinds of storages:
-
Data Directory - This is a location used by the app and only the app uses it exclusively. The app will store say attachments, profile pics, avatars, plugin code and whatever it needs in this directory. Data directory has to be ext4 because the packaging code requires file system permissions to work etc. This is the location you specify in https://docs.cloudron.io/storage/#app-data-directory . You can think of this app as part of an app's "state" (just like the database) and it is thus backed up and restored.
-
External Directory/Volumes - This location can be any file system. Mostly the app just reads file (docs, music, movies etc) from this. So the permissions don't matter. It is also expected that these directories can be shared across apps. This is similar to the home directory on linux or My Documents on Windows. It's just separate directory that apps have access to. If you uninstall, backup apps, that folder is untouched. For these kind of directories, the file system type is not relevant. It can be anything - sshfs, cifs, ext2 whatever.
-
-
@girish OK, thanks, that's clearer.
However, some of the music streaming apps etc like Navidrome (which is what I thought I'd use a Hetzner Storage box CIFS for) can have users with different permissions too, so still not sure if that'll be possible?
Although the way I'd actually want to use it is that only one person (me) will be able to upload music, and all other users will just have read access, so I guess that could work fine?
-
@jdaviescoates said in What's coming in 6.0 (take 2):
However, some of the music streaming apps etc like Navidrome (which is what I thought I'd use a Hetzner Storage box CIFS for) can have users with different permissions too, so still not sure if that'll be possible?
Yes, definitely possible. App users are entirely different from file system permissions. Without getting too technical, users inside an app are just "virtual users". They just exist inside the app's database.
-
@girish OK!
So maybe I will start uploading my 290GB of music up to my new 500GB Hetzner Storage Box to use like this as soon as 6.0 goes live!
-
@girish said in What's coming in 6.0 (take 2):
see the WIP tag
Oooh, just looked at that for the first time in a while and I see BigBlueButton and Jitsi Meet are included... who is actively working on them atm, is it you @girish @nebulon ?
Cloudron 6.0 with full text email search, volumes and BBB/ Jitsi would be SO awesome! (loving all the new music/ media apps too!)
-
Once there are proper Volumes, my River app won't have too much reason to exist, so you can expect a few other apps soon after 6.0 release as I split river into independent apps (namely SickChill, CouchPotato, and Transmission)
-
@mehdi said in What's coming in 6.0 (take 2):
Once there are proper Volumes, my River app won't have too much reason to exist, so you can expect a few other apps soon after 6.0 release as I split river into independent apps (namely SickChill, CouchPotato, and Transmission)
Did your River app combine those apps with an added external storage connection bit?
-
@lonk It combines Jellyfin (for which there's already an app now), Transmission, SickChill, Couchpotato (the 3 I just mentioned), a custom file manager (built before Cloudron had one), and a custom TV Shows and Movies streaming interface (built before I integrated Jellyfin). Plus a few custom things, like a script to auto-remove finished torrents on transmission, and a script to auto-convert videos to MP4 for easy streaming in the browser (for my custom streaming interface. I disabled it now that I mainly use jellyfin, which handles this automatically).
-
@mehdi said in What's coming in 6.0 (take 2):
Once there are proper Volumes, my River app won't have too much reason to exist, so you can expect a few other apps soon after 6.0 release as I split river into independent apps (namely SickChill, CouchPotato, and Transmission)
When you have time, please split them out and I can start approving them as unstable already. It also gives me a good test bed to test the volumes stuff across apps and permissions.
-
@girish said in What's coming in 6.0 (take 2):
When you have time, please split them out and I can start approving them as unstable already.
OK, will do. I will have to pick your brain a little about how to handle authentication for these apps. In river, they're behind a custom auth proxy, as they don't handle auth by themselves.
-
@mehdi said in What's coming in 6.0 (take 2):
OK, will do. I will have to pick your brain a little about how to handle authentication for these apps.
Would it make sense to bring that into box code somehow? Maybe we add some flag in the manifest to turn on this "authentication wall".
@nebulon Is it possible to have a login screen like the surfer app but served from box code? I guess session management possibly won't work since app has no clue about this? Maybe logout won't work as well since there is no logout button in the app.
-
@girish said in What's coming in 6.0 (take 2):
Would it make sense to bring that into box code somehow? Maybe we add some flag in the manifest to turn on this "authentication wall".
It would be possible yeah. An other possibility would be to have a standard way to do it in the "base image", to make it easy to implement in the apps that need it. I would probably go with the "base image" path, because it would be easier to configure on an app level IMO.
-
@mehdi Good idea. Maybe then for the moment we can copy/paste apache configs in every app (like https://git.cloudron.io/cloudron/simple-torrent-app/-/blob/master/apache/cloud-torrent.conf ) for now and once we have a something common, I can try to put it in the next base image.
-
I really like that approach as well. I'll follow what you guys create and what changes occur to the
baseimage!
️ -
So the idea is to put some kind of "framework" into the base image, which can be used by apps? Wouldn't that anyways still not mean that an app needs to be patched for at least the logout action? Also would we do this as a php set of features? I do like to not pull this into the platform code as such, as that does not increase dependency on that.
Alternately, we could certainly add a login screen served up with some kind of session. The question then, as already mentioned, is how to logout. We could provide the app with a logout link, still that needs patching the app to some extent.
-
@nebulon IMO the login part is much more important than the logout part. We can even do completely without the logout at all in the interface, with just a
/logoutURL that one would have to enter manually (if ever).As to the precise tech to use, I already have a working version in Node.JS in river that I could isolate. If you guys prefer to re-do it in PHP instead or something else, it's your choice.
-
Right, I was only bringing up php since that might be more commonly already be used within such apps, I would prefer a nodejs solution though. Maybe we can collect some arguments for and against adding to the base image or into the platform.
If we add it to the platform, we could have it more easily streamlined with the Cloudron look and feel, however within the base image the app could style it more towards its own look and feel. Putting it in the platform on the other hand would allow support translation now once it fully landed. Also say we use a nodejs based version, then we have to keep running an additional process with possibly another proxy even?
-
Just pointing out that this might be better served by using something like a proper gateway and load balancing solution in front of the apps like Kong rather than NGINx on the box. An add-on for a basic auth screen could just be a config tweak to the box ingress which hits an auth wall - added advantage would be much more flexible routing to apps (sub-path, etc.) and flexible options like having multiple DNS names resolving to the straight A record for things like apps that can serve multiple domain names off one instance.
-
Bringing in some service mesh concepts would be interesting.
-
@jimcavoli
You mean to use Kong or Traefik (this are exeple) as incoming proxy for every container ?If yes, we need to consider how reliable is Nginx and the feature that can be easily added (Proxy_Cache, brotli, WAF, ...)
Matteo. R.
Founder and Tech-Support Manager.
MooCloud MSP
Swiss Managed Service Provider -
@moocloud_matt yeah, there's currently a
boxlevel NGINx proxy - my idea is to replace that with a very hand-wavy something else which may be capable of shimming authentication for those things that just don't have it inbuilt (Kong) or if we go a different route on that sort of thing, we could use Traefik or similar at that layer. I think the use cases are intriguing enough to at least try the Kong route and fall back to something like Traefik if need be -
@jimcavoli
for adding .htpasswd support ? or what kind of auth are you talking about?Matteo. R.
Founder and Tech-Support Manager.
MooCloud MSP
Swiss Managed Service Provider -
@moocloud_matt Kong is a pretty expansive topic on its own, but the idea would be with it in charge of ingress/routing, you could have a simple add-on config that would take care of enabling a plugin like https://docs.konghq.com/hub/kong-inc/ldap-auth/ on the route to a particular app, so you could have HTTP basic auth, but completely backed by the full Cloudron user store for any app that doesn't have its own scheme, providing clean headers that could be easily handled by that app's eb server or whatever
-
Kong is indeed a separate complex topic. I think for the moment, if we had some template that people can quickly copy over to the app to get auth screen/login, it will help already. @nebulon do we have such a template already ? (like the one we use for our internal apps).
-
@jimcavoli
i think that's possible with nginx too, the ldap backend for auth.
Custom Template for nginx config, will be the best i think, especially for performance optimization.But this Kong proxy is interesting i will ask to my team, if they have use it.
Matteo. R.
Founder and Tech-Support Manager.
MooCloud MSP
Swiss Managed Service Provider -
@girish said in What's coming in 6.0 (take 2):
Kong is indeed a separate complex topic. I think for the moment, if we had some template that people can quickly copy over to the app to get auth screen/login, it will help already. @nebulon do we have such a template already ? (like the one we use for our internal apps).
Yes in various shapes, but all nodejs based. This would be trivial to add, however if many of those apps are just apache+php does it make sense to add supervisor+nodejs+someproxy to those apps just for a login screen?
-
@girish quick question in the mailbox sharing feature. If you make it so that a single inbox can have multiple owners (great feature btw), do you think it would then be possible to have the option to set a group as the owner so the mailbox ownership gets dynamically updated with changes in group membership? Thanks
-
@avatar1024 Yes, that's the idea. The ownership will be dynamic.
That said, the initial outlook for the feature is not looking so good. There are two issues that need to be sorted out (suggestions/ideas welcome):
-
Apps like SOGo show the "display name" of the user in the main UI. With a shared mailbox, it's not clear where this name should come from. With a single user, we give SOGo, the user's name. With multiple users, it's not clear what this should be.
-
The authentication (from a user's point of view) is a bit confusing. Or maybe it's not, I would welcome some feedback here. You have to authenticate with the user's username/password but use the shared mailbox as the mailbox name. In some ways, this is the case already, when you use a different mailbox name with a different username.
-
-
@girish thanks for the reply.
Perhaps there are a few things I do not understand about the technical implementation of such a feature but I will still try to give my opinion. As you describe in the first post on this thread the idea is, rather that a "shared mailbox" as such, to have a single mailbox with multiple owners, so:
-
for point 1) I suppose you mean that the name of the "owner" is fed to the app when the owner is assigned to the mailbox rather than when the user logs in? If so, would it possible to feed the name of the user that logs in into that (which will be recognised through its unique pair of credential: mailbox name + its unique password). If not then I'd say, as a first implementation of this feature (which could be improved later), then the name of the mailbox (as in the prefix before the @) should be fed to the app as the Display Name, when more than one owner are set to that mailbox
-
for point 2, then yes, as you say the username (for login) should just remain the full mailbox email address (just that in the case of a shared one, one username will but associated to multiple passwords as valid login credentials).
-
-
@girish Or as I suggested in my previous post, if we could assign a "group" as the owner of a mailbox, then group name could be fed as the Display Name (for point 1) and you potentially use the group name for the login username (for point 2).
@avatar1024 said in What's coming in 6.0 (take 2):
...be possible to have the option to set a group as the owner...
-
@avatar1024 said in What's coming in 6.0 (take 2):
@girish Or as I suggested in my previous post, if we could assign a "group" as the owner of a mailbox, then group name could be fed as the Display Name (for point 1) and you potentially use the group name for the login username (for point 2).
Making shared mailbox feature work only with groups is an excellent idea! I have to try this out and get back on how well this works.
-
@moocloud_matt said in What's coming in 6.0 (take 2):
@jimcavoli
i think that's possible with nginx too, the ldap backend for auth.
Custom Template for nginx config, will be the best i think, especially for performance optimization.But this Kong proxy is interesting i will ask to my team, if they have use it.
Should we branch this topic off into its own thread about centralized authentication? It seems like an important aspect to discuss but this thread is about 6.0, which this couldn't be a part of, right? This would be further down the road? @girish @nebulon
-
@girish Glad it was at least a useful suggestion. Let's see if it works in practice
-
@avatar1024 That worked out quite nicely. You can now select a group as owner of a mailbox. founders and sales-team are groups in the screenshot below.
-
@girish Will each person's password work for IMAP/SMTP? (users in the group)
We're not here for a long time - but we are here for a good time :)
Jersey/UK
Work & Ecommerce Advice: https://brandlight.org
Personal & Software Tips: https://marcusquinn.com -
@marcusquinn Yes, that's the idea. Each user can use their own password for sending/receiving with same mailbox.
-
@girish Just a suggestion, but IMO it should have two dropdowns like the access section for apps. One for users and one for groups. Unsure of the implementation details but that just stood out to me.
-
@atrilahiji Currently, one can only choose a user or a group. This was just the initial UI anyway for testing the backend. I think we probably need select box with groups like
