What's coming in 6.0 (take 2)
-
Funny enough, when I migrated from a VPS to on-prem (my house) I initially tried installing it on 20.04. Didn't work. :^)
-
jdaviescoatesreplied to girish on Sep 29, 2020, 12:40 AM last edited by jdaviescoates Sep 29, 2020, 12:58 AM
@girish said in What's coming in 6.0 (take 2):
Ubuntu Focal 20.04 support
Just to check, but I assume Ubuntu 18.04 will continue to be supported by Cloudron until it's end of life (i.e. until April, 2023)?
@girish said in What's coming in 6.0 (take 2):
- Mail - Full text search via IMAP (solr integration)
Can't wait for this! I think this will mean I'll finally have the courage to ditch G Suite for my primary email!
But, I wonder, will it somehow enable (at least some) of the search operators available in Gmail?
From the little I've read, I get the impression solr can do this stuff, but I just wonder if there will be (or are?) any open source webmail clients that can actually take advantage of it once we've got it?
-
@jdaviescoates yes ubuntu 18.04 will continue to be supported.
-
necrevistonnezrreplied to jdaviescoates on Sep 29, 2020, 9:33 AM last edited by necrevistonnezr Sep 29, 2020, 9:52 AM
@jdaviescoates said in What's coming in 6.0 (take 2):
From the little I've read, I get the impression solr can do this stuff, but I just wonder if there will be (or are?) any open source webmail clients that can actually take advantage of it once we've got it?
Apparently, Roundcube should work (https://wiki.magenbrot.net/linux/mail/dovecot/volltextsuche_in_mails_mit_solr (German) and https://github.com/RainLoop/rainloop-webmail/issues/73#issuecomment-251093397).
Rainloop works with SOLR and dovecot (which I think Cloudron uses) if you create a virtual mail folder with all mail: https://github.com/RainLoop/rainloop-webmail/issues/73#issuecomment-412377275
Since the otherwise excellent Rainloop doesn't even support search in multiple folders, this would be a great improvement! I hope wish that @girish and @nebulon would / could preconfigure dovecot accordingly...
-
@necrevistonnezr thanks, that all sounds positive, but I don't see anything there about which search operators will work. Guess maybe I'll just have to wait and see!
-
Some big news: We will be adding i18n support in Cloudron 6. This work is being sponsored by Medias-Cite and their Cloudron project OSINUM. Thanks @Benoit ! As part of the work, we will also package the Weblate app (for translations).
-
@girish said in What's coming in 6.0 (take 2):
@ruihildt You can/should still do it !
Something to announce ?
-
We got some inquiries about what languages we will support. Currently, it's early days and we are just deciding on how to go about internationalization. Usually, translating an app is 2 steps. Step 1 is i18 phase where the app is made translation friendly. Step 2 is the l10n phase where localizations are provided.
At this point, all we know is that we will atleast launch with french and german translations.
-
Some updates on volume management. There is now a volume view.
You add a host path as a volume and give it a name. This is most likely some EBS/external hard disk/Block Storage. There is also the file manager integration (the button next to the delete button).
You can then go to the app view and mount these volumes into the apps. There is a new Storage section.
There is a section overload in the app configure view We are looking to reducing the list of sections in the left pane. It's making things hard to find.
-
This looks wonderful!
Off the top of my head, I could see the Console + Repair sections being merged and the Resources + Access Control + Location sections being merged. But honestly I am struggling to come up with names so I suppose thats why you have them the way you have them now.
-
@girish Amazing!
It's almost like you develop with the power of telepathy sometimes
-
@girish Any chance of you adding a button in for resizing the partition after upgrading a VM?
-
@marcusquinn Not for this release but hope to implement more volume management tools (like mounting nfs/sshfs, backing up volumes, resizing etc) as we go.
-
Not a feature I was expecting but I know I'll take advantage of it. This is pretty cool!
-
@marcusquinn said in What's coming in 6.0 (take 2):
@girish Any chance of you adding a button in for resizing the partition after upgrading a VM?
+1
Resizing my backup volume on Hetzner is one of the few reasons I have to occasionally ssh into my VPS.
-
Externalizing a volume to a Minio/S3/Object store would be nice.
-
@robi
s3 filesystem are not so mature, i read a lot on them because use s3 with dovecot in our mail server and the performance are terrible in u use s3 as FS, 20/40 ms depending on where you host the s3 server.The optimal solution is to use plugin or software that integrate s3 in to the App/script that you want to use.
-
@girish I could not have imagined a better system ! Looks amazing !
Three questions:
1/ How do you decide where in the app these are gonna be mounted? Is it in the same path for all apps ? Or could I create a given volume, and mount it at
/app/data/toto
in app 1 and/app/data/tata
in app 2 ?2/ How does it work when there already is data in the app's original
/app/data
before you mount the volume to it? Is it just hidden as long as the volume is mounted?3/ How does it work with regards to backups?
-
For 1 and 2: Those volumes will be mounted into the apps based on
/media/<volumename>
and thus one volume associated with apps will always have the same mountpoint within the apps.For 3: Initially those won't be backed up at all, but we will likely add optional backup per volume as well. The user has to be aware of the fact that volumes are not part of the app backups as such!
-
@nebulon said in What's coming in 6.0 (take 2):
For 1 and 2: Those volumes will be mounted into the apps based on
/media/<volumename>
and thus one volume associated with apps will always have the same mountpoint within the apps.For 3: Initially those won't be backed up at all, but we will likely add optional backup per volume as well. The user has to be aware of the fact that volumes are not part of the app backups as such!
That’s the difference to storage mounted in „Resources“ - this will get backed up, correct?
-
The volumes are setup outside of the apps first and are essentially independent. Once you've created a volume you can then mount that into an app via the Storage section in the app configuration. Even if mounted though, it will not be part of the app backup, since the main reason for volumes is to be mounted into multiple apps. For that we have to first figure out how to manage those with regards to backups.
-
d19dotcareplied to nebulon on Oct 31, 2020, 5:09 PM last edited by d19dotca Oct 31, 2020, 5:17 PM
@nebulon That makes sense. I’m of the opinion that optional block storage should effectively be backed up at the host level not the application level. In other words, taking a snapshot or something like that which many providers allow even volumes / block storage to be snapshotted. Much like how we need to snapshot our actual VPS on occasion too for a backup strategy of the entire server.
-
@nebulon said in What's coming in 6.0 (take 2):
The volumes are setup outside of the apps first and are essentially independent. Once you've created a volume you can then mount that into an app via the Storage section in the app configuration. Even if mounted though, it will not be part of the app backup, since the main reason for volumes is to be mounted into multiple apps. For that we have to first figure out how to manage those with regards to backups.
No, I mean external volumes that you can currently already move the data dir to under „Resources“ in the app settings - this gets backed up, correct?
-
@MooCloud_Matt this is only true if you use dumb S3 connectors that treat it like a filesystem instead of an object store.
It's a very different experience when things are treated properly as objects.
-
Re: Volumes - I think this will create a solution for something that I have been trying to resolve - i.e. Nextcloud app backups WITHOUT the data!
I want to be able to back up the app via Cloudron natively - but backup the data via borg/restic/kopia or the like. This looks like it could resolve this issue if the data was an attached volume. Very much looking forward to this feature if it's like that.
-
@iqweb said in What's coming in 6.0 (take 2):
Re: Volumes - I think this will create a solution for something that I have been trying to resolve - i.e. Nextcloud app backups WITHOUT the data!
Yes, I think this will solve that use case. You mount a volume into nextcloud and configure nextcloud to use it as external storage.
-
@girish can you also mount an ‘on server’ folder like that?
-
@imc67 Yes. Currently, the main restriction is that you can only add host paths under /mnt and /media as a volume. So, you can create some NFS mount or SSHFS mount there and then add it as a volume and then mount it into an app.
-
@girish can't wait to have volumes to play with! what's your guestimate ETA?
-
@jdaviescoates Cloudron 6 is ~2-3 weeks away. The translation project is quite a massive change, so maybe I am being overlay optimistic here.
-
@girish all power to your elbow! (i.e. good luck!)
-
@scooke said in What's coming in 6.0 (take 2):
@ruihildt Your comment from 26 days ago, "As an aside, they are advertising Cloudron as free software, which is not correct at the moment." I came across it as I read through the entire thread.
I think you've missed the point
@ruihildt was talking about free as in freedom (which is how that project advertises Cloudron), not free as in price.
-
scookereplied to jdaviescoates on Nov 2, 2020, 2:54 PM last edited by scooke Nov 2, 2020, 3:02 PM
@jdaviescoates I figured so. But I wonder why @ruihildt bothered to point out what he did. The few lines I found on the website aren't actually advertising Cloudron as %100 open source, but rather that the code they use is open source. I'd be more bothered by their approach, like Framasoft, where if the user has no idea about the existence of Cloudron, in this case, the company's description of what they do (https://osinum.fr/) makes it sound like it is and has been all their own doing. Much of what they promise to do is based on whether Cloudron, or all the other myriad open source projects in use, has or will implement it, but they don't clearly specify that. If I found them, and started using them, and THEN discovered they actually use Cloudron, I'd be a little miffed, and would just switch over to Cloudron.
Anyway, I have no clue what "being sponsored by Medias-Cite" means, but if the Cloudron team are fine with it, and this "sponsoring" helps move Cloudron forward, great.(A bit more reading shows that they, osinum.fr, are charging a minimum 150 EURO/month!!! I guess they are paying Cloudron for the Premium Plus plan. They offer, for almost triple the price of Cloudron's pricing,
1 domain name (.fr, .com, .org, .net)
Accommodation in France
Shared server
100 GB
5 applications
50 users
Management of user groups
Daily backups
Surveillance / Monitoring / Security
Email supportMaybe I should get into offering cloudron!!
-
jdaviescoatesreplied to girish on Nov 2, 2020, 7:32 PM last edited by jdaviescoates Nov 2, 2020, 9:41 PM
@girish fair enough
Just to quickly say (sorry) to @scooke re:
the user has no idea about the existence of Cloudron
I'm not sure where you are looking but on https://osinum.fr/ there is this:
Which translates as:
"The core of OSINUM is the Cloudron software and its team of developers."
So 1) they seem quite clear about being based on Cloudron, and 2) they've actually changed what it used to say which @ruihildt was commenting on.
Previously it said: "The core of OSINUM is the open source software Cloudron and its team of developers."
Edit: see my screenshot here https://twitter.com/jdaviescoates/status/1314243996441620480
/end.
-
@jdaviescoates @girish D'accord.
-
jdaviescoatesreplied to girish on Nov 3, 2020, 3:53 PM last edited by jdaviescoates Nov 3, 2020, 3:53 PM
@girish said in What's coming in 6.0 (take 2):
You add a host path as a volume and give it a name. This is most likely some EBS/external hard disk/Block Storage.
I'm assuming volumes will support CIFS unlike current external data storage, right? (if not I had better cancel the Hetzner storage box I just ordered for this purpuse (or just start using it for backups where CIFS is already allowed)! )
-
@girish said in What's coming in 6.0 (take 2):
@imc67 Yes. Currently, the main restriction is that you can only add host paths under /mnt and /media as a volume. So, you can create some NFS mount or SSHFS mount there and then add it as a volume and then mount it into an app.
I really would appreciate it if you could create a step by step instruction in the docs by that time. I guess more users with large base “disk” would want to use this!
-
@jdaviescoates said in What's coming in 6.0 (take 2):
I'm assuming volumes will support CIFS unlike current external data storage, right? (if not I had better cancel the Hetzner storage box I just ordered for this purpuse (or just start using it for backups where CIFS is already allowed)! )
The underlying problem is still there, CIFS / SMB don't support users and groups in the FS, so i guess it will depend on the apps, if they rely on FS permissions or not
-
@mehdi hmz, thanks, so I'm guessing likely not then.
So Hetzner storage boxes likely a no no for volumes (I guess I'll cancel the one I've just ordered, or just use it for backups)
I wonder what would be a good option...
I guess maybe Hetzner Volumes would work well, but they cost a lot more (a 500GB storage box is 5.68€/mo, whereas a 500GB Volume is 24€/mo)
-
@jdaviescoates said in What's coming in 6.0 (take 2):
So Hetzner storage boxes likely a no no for volumes (I guess I'll cancel the one I've just ordered, or just use it for backups)
So, there's two kinds of storages:
-
Data Directory - This is a location used by the app and only the app uses it exclusively. The app will store say attachments, profile pics, avatars, plugin code and whatever it needs in this directory. Data directory has to be ext4 because the packaging code requires file system permissions to work etc. This is the location you specify in https://docs.cloudron.io/storage/#app-data-directory . You can think of this app as part of an app's "state" (just like the database) and it is thus backed up and restored.
-
External Directory/Volumes - This location can be any file system. Mostly the app just reads file (docs, music, movies etc) from this. So the permissions don't matter. It is also expected that these directories can be shared across apps. This is similar to the home directory on linux or My Documents on Windows. It's just separate directory that apps have access to. If you uninstall, backup apps, that folder is untouched. For these kind of directories, the file system type is not relevant. It can be anything - sshfs, cifs, ext2 whatever.
-
-
@girish OK, thanks, that's clearer.
However, some of the music streaming apps etc like Navidrome (which is what I thought I'd use a Hetzner Storage box CIFS for) can have users with different permissions too, so still not sure if that'll be possible?
Although the way I'd actually want to use it is that only one person (me) will be able to upload music, and all other users will just have read access, so I guess that could work fine?
-
@jdaviescoates said in What's coming in 6.0 (take 2):
However, some of the music streaming apps etc like Navidrome (which is what I thought I'd use a Hetzner Storage box CIFS for) can have users with different permissions too, so still not sure if that'll be possible?
Yes, definitely possible. App users are entirely different from file system permissions. Without getting too technical, users inside an app are just "virtual users". They just exist inside the app's database.
-
@girish OK!
So maybe I will start uploading my 290GB of music up to my new 500GB Hetzner Storage Box to use like this as soon as 6.0 goes live!
-
@girish said in What's coming in 6.0 (take 2):
see the WIP tag
Oooh, just looked at that for the first time in a while and I see BigBlueButton and Jitsi Meet are included... who is actively working on them atm, is it you @girish @nebulon ?
Cloudron 6.0 with full text email search, volumes and BBB/ Jitsi would be SO awesome! (loving all the new music/ media apps too!)
-
Once there are proper Volumes, my River app won't have too much reason to exist, so you can expect a few other apps soon after 6.0 release as I split river into independent apps (namely SickChill, CouchPotato, and Transmission)
-
@mehdi said in What's coming in 6.0 (take 2):
Once there are proper Volumes, my River app won't have too much reason to exist, so you can expect a few other apps soon after 6.0 release as I split river into independent apps (namely SickChill, CouchPotato, and Transmission)
Did your River app combine those apps with an added external storage connection bit?
-
@lonk It combines Jellyfin (for which there's already an app now), Transmission, SickChill, Couchpotato (the 3 I just mentioned), a custom file manager (built before Cloudron had one), and a custom TV Shows and Movies streaming interface (built before I integrated Jellyfin). Plus a few custom things, like a script to auto-remove finished torrents on transmission, and a script to auto-convert videos to MP4 for easy streaming in the browser (for my custom streaming interface. I disabled it now that I mainly use jellyfin, which handles this automatically).
-
@mehdi said in What's coming in 6.0 (take 2):
Once there are proper Volumes, my River app won't have too much reason to exist, so you can expect a few other apps soon after 6.0 release as I split river into independent apps (namely SickChill, CouchPotato, and Transmission)
When you have time, please split them out and I can start approving them as unstable already. It also gives me a good test bed to test the volumes stuff across apps and permissions.
-
@girish said in What's coming in 6.0 (take 2):
When you have time, please split them out and I can start approving them as unstable already.
OK, will do. I will have to pick your brain a little about how to handle authentication for these apps. In river, they're behind a custom auth proxy, as they don't handle auth by themselves.