Changes to WordPress apps

girish

There was some good discussion (thanks @jdaviescoates !) about how to best handle the two WordPress packages. We are making some changes thanks to the suggestions there.

Our primary motivation here is to just make Cloudron more WordPress friendly and also make it easier for a user to choose the correct app package. I think we can all also agree the following are needed:

Make WordPress plugins should just work
Make migrations/imports from existing sites just work. Sadly, many of the migration plugins bring in source code and not just data.
Make many of the security plugins which do all sorts of crazy things like adjust the admin URL, modify files etc work. While we personally don't "vouch" for such security practices, we can't deny that WP is still the most installed app in our platform and most people install these plugins. In the spirit of picking our battles, we concede this one to the existing WP ecosystem
Clarify expectations up front when installing the WP app.

With the above in mind: It's clear that it can only be supported in Unmanaged WordPress realistically. But "Unmanaged" as a word has a bad connotation. So, we will rename WordPress (Unmanaged) to WordPress (Developer) (thanks to @Lonk for the suggestion). We can keep the current WordPress (Managed) with the same name since it indicates it's like a managed service with subset of working plugins. This is similar to any other managed hosting WP service where they have lots of restrictions.

Currently, we don't want to remove WordPress (Managed) as such. It still has strong security benefits.

Feature Parity

We will make both the apps be the same feature wise. The only main difference will be whether you can edit core WP updates and how you do WP updates (you can read more text in the DESCRIPTION below). So, we will have the following in both:

LDAP
Optional redis, we will make this part of Cloudron 6.0
SFTP access. For security reasons, this will come with a checkbox that has to be enabled for non-admins to use SFTP in LDAP mode.
WP CLI
Plugin auto-update feature work (which is part of WordPress).

DESCRIPTION changes

We will change the description file for both apps. This is the text that pops up when you install an app (not the postinstall, but in the appstore view).

WordPress (Managed) - See https://git.cloudron.io/cloudron/wordpress-managed-app/-/blob/master/DESCRIPTION.md

WordPress (Developer) - See https://git.cloudron.io/cloudron/wordpress-developer-app/-/blob/master/DESCRIPTION.md

Lonkle

I’m honored you chose my suggestion! I’m excited about feature parity between the two.

Would you like any help with LDAP integration for Unmanaged? I just coded a small PHP library for doing so but you mentioned security worries in Wordpress (Developer). If you open another thread, the community can talk about ways to do it securely, but you’d know best about how to do so since you guys made the Managed Version already integrate with LDAP.

girish

@Lonk said in Changes to WordPress apps:

Would you like any help with LDAP integration for Unmanaged?

Thanks for the help but it's really just copy/pasting from one app to the other Hopefully, I can get all the changes done by end of today.

Lonkle

@girish Also, I read the description for both versions and your rewrites are both perfect. ️

PS. You forgot to write Multisite support for the Developer Edition - I’m still going to be writing box changes to support the slight database peculiarities between single site and Multisite so you can change the URL from Cloudron and I’ll be updating your WP-CRON script to work with all sub-sites (if they exist, it will detect if it’s a Multisite installation).

Lonkle

@girish said in Changes to WordPress apps:

@Lonk said in Changes to WordPress apps:

Would you like any help with LDAP integration for Unmanaged?

Thanks for the help but it's really just copy/pasting from one app to the other Hopefully, I can get all the changes done by end of today.

Woah, that quick! Did you give up on your security worries of Unmanaged LDAP because it’s called Developer Edition or did you solve them?

girish

@Lonk said in Changes to WordPress apps:

You forgot to write Multisite support for the Developer Edition

Hopefully, we can add multi-site to both. I have never used a multi-site WP, so I don't know if it's compatible or not with the read-only file system.

Lonkle

@girish said in Changes to WordPress apps:

@Lonk said in Changes to WordPress apps:

You forgot to write Multisite support for the Developer Edition

Hopefully, we can add multi-site to both. I have never used a multi-site WP, so I don't know if it's compatible or not with the read-only file system.

It’s compatible with a read only file system. But it would have to be chosen during installation or there would have to be an option in Wordpress Configure within Cloudron to “Convert to Multisite” but I’m more in favor of someone choosing Multisite during installation.

Multisite just adds more databases (which is fine in read only) and more folders in /upload (which is fine in read only).

girish

@Lonk said in Changes to WordPress apps:

Woah, that quick! Did you give up on your security worries of Unmanaged LDAP because it’s called Developer Edition or did you solve them?

Ah right, forgot to mention this. There is no real fix for this. SFTP access when given to normal Cloudron users to edit code..is a security problem. I think we will put a note in the docs/tooltip warning in the UI. The user has to evaluate if this is really a security issue for them. We were thinking we will add a checkbox which user has to click to allow SFTP access for non-admins.

girish

@Lonk said in Changes to WordPress apps:

. But it would have to be chosen during installation or there would have to be an option in Wordpress Configure within Cloudron

Ah, it's an install time flag for WordPress? In general, is it possible to convert multi-site to single-site and vice versa (in an automated way) ?

Lonkle

@girish No no, not an install time flag. You can convert at any time from single-site to multisite. But there's no converting back once you're on multisite.

I only mentioned that you might want to do it during install time because then you wouldn't have to add an option in dashboard asking a user if they'd like to "Convert to Multisite (This Cannot Be Undone)" kinda thing. But doing it at install time, or after install time - just as easy.

Because I had SFTP access on the Developer Edition, I was able to just enable it manually. But that does come with caveats - like you cannot change the URL in Cloudron if you're a multisite (it's stores the domain in a different place in the DB with multisite, and also in wp-config.php). That, and the WP-CRON that would need multisite support.

The only think beyond those two things (which I could code easily to integrate with Cloudron) is real domain aliases (not redirections) would be needed. Which, after looking into reverseproxy.js, shouldn't be hard. I just am horrible at editing dashboard code every time I try.

robi

what is the purpose of including redis?

d19dotca

@robi Redis can be used for caching improvements to WordPress, some plugins will work with redis as the caching backend.

robi

@d19dotca there's already several instances that run as shown in Cloudron services, but I can't tell if that's shared or not.

none of the other databases show which apps use them.

girish

@robi The databases are shared since MySQL, PostgreSQL, Mongo etc are multi-tenant. You can setup rules to isolate users and apps. Redis is not multi-tenant. For this reason, there is a redis instance per app. Redis is ultra-light weight though, so it's not really a resource hog/issue.

marcusquinn

I spotted an article recently that reminded me of this debate, and I think I can make the distinction a little more familiar:

Wordpress (Managed) is more like wordpress.com
Wordpress (Unmanaged) is more like wordpress.org

robi

@girish thanks for that.
is that something we can fix graphite with?

Lonkle

@girish said in Changes to WordPress apps:

@robi The databases are shared since MySQL, PostgreSQL, Mongo etc are multi-tenant. You can setup rules to isolate users and apps. Redis is not multi-tenant. For this reason, there is a redis instance per app. Redis is ultra-light weight though, so it's not really a resource hog/issue.

I had been wondering this entire time why Redis wasn't shared (I used LAMP a lot in my testing as did I docker network inspect cloudron so because you name the containers as hashes, the redis ones made it easier to identify which was which (my VPN client which had no redis, and the LAMP client), so that was pretty useful to me funnily enough.

Lonkle

@marcusquinn said in Changes to WordPress apps:

I spotted an article recently that reminded me of this debate, and I think I can make the distinction a little more familiar:

Wordpress (Managed) is more like wordpress.com

Wordpress (Unmanaged) is more like wordpress.org

Great comparison and should hit home, I - for no particular reason - think the name should stay, but at the end of each desciption, it could say:

"Think of this as as simple and easy "wordpress.com" but with plugin support."

Something to that affect would work to compare the two for existing Wordpress users looking to switch.

marcusquinn

@Lonk The worry with wordpress.com, Shopify etc, is always the risk of de-platforming with no recourse, when they are judge & jury.

I'm convinced self-hosting federation is the future. Compared to relying on the whims of a single provider, that can have their abuse processes abused by rogue competitors is a very real risk.

These risks increase the more successful you are when rogue competition or trolls can take a dislike and interrupt innocent people & businesses at any time.

Lonkle

@marcusquinn said in Changes to WordPress apps:

@Lonk The worry with wordpress.com, Shopify etc, is always the risk of de-platforming with no recourse, when they are judge & jury.

I'm convinced self-hosting federation is the future, relying o the whims of a single provider, that can have their Abuse processes abused by rogue competitors is a very real risk, that increases the more successful you are and rogue competition or trolls can take a dislike and interrupt innocent people & businesses at any time.

Your belief in open source is one of the many reasons I respect ya, man. I think it's more important than people think or realize. Even I didn't for a lonk time. But it's important to own your future. Understand what you're running on, be able to fix something that the developer doesn't have time to because you have access to what you're running on. But seriously, I gotta brag about Cloudron's API one more time, it's amazinggggg. And half of it's private! I'll think about documenting it one day though if I find there's interest but it seems like I've been the only one interested as much in the API.

marcusquinn

@Lonk Yeah, all the devs I work with go quiet when I tell them we're going to open-source years and millions of dollars of development work.

I believe in OS, making it happen just takes longer than even my ideologies hope for.

Lonkle

@marcusquinn said in Changes to WordPress apps:

@Lonk Yeah, all the devs I work with go quiet when I tell them we're going to open-source years and millions of dollars of development work.

I believe in OS, making it happen just takes longer than even my ideologies hope for.

I'm an idealists. I was telling @girish in another thread - as long as you want to do that and you plan on doing it (we were talking about a box patch) "I don't care about when". This is obviously just my own opinion. But I feel like you're contributing ideologically. It's not the actual action of doing so, it's the intention, even if you don't succeed. At least, that's how I feel. Plus, I'm pretty patient. One day is a perfect answer. Life is crazy and hectic and ever changing.

marcusquinn

@Lonk Watch this space - we'll make WP & Woocommerce a seriously fast and international enterprise app that anyone can build on!

It all started because we created what we wanted to use for our needs but did it in a way that should serve almost any.

Lonkle

@marcusquinn said in Changes to WordPress apps:

@Lonk Watch this space - we'll make WP & Woocommerce a seriously fast and international enterprise app that anyone can build on!

It all started because we created what we wanted to use for our needs but did it in a way that should serve almost any.

Oh, you've def piqued my curiosity. I'll be watching the space for sure. I've been needing to get back into the WP community (specifically Post Status was my favorite Wordpress Community) anyway so I'll start paying attention more.

marcusquinn

@Lonk Did you try our plugin unloading plugin? That should give you some serious TTFBs

marcusquinn

We've just switch https://healthshop.net to run on Cloudron. 6,000 products, multilingual, multi-everything — and I know we still have a bunch more optimisations to go.

If ever anyone says WP & Woo can't scale, send them my way

Looking forward to getting into more Cloudron App optimisations.

Honestly, must have tried every other WP hosting on the planet by now - and none of them are truly optimised or even transparent about what they consider optimisation beyond their caching plugin masquerades.

Performance is about uncached code optimisation, and I still can't imagine any of the other so-called enterprise ecommerce platforms being able to do what we can with good old WP and it's community.

Lonkle

@marcusquinn said in Changes to WordPress apps:

If ever anyone says WP & Woo can't scale, send them my way

I've been talking about the viability of Woo and Scaling for awhile so I'm excited to see what you've done (at a higher level than your normal communication about it) when you announce it fully. ️

jdaviescoates

@Lonk said in Changes to WordPress apps:

The only think beyond those two things (which I could code easily to integrate with Cloudron) is real domain aliases (not redirections) would be needed.

I'd love to see WordPress Multisite on Cloudron.

As I understand it (one of) the problem(s) is that I don't think it's currently possible to have multiple domains pointing to the same Cloudron app, and imho we'd need that to work for Multisite to be useful (so that all the subsites within the multisite can be mapped to their own TLD).

I guess that is what you mean by real domain aliases?

marcusquinn

@Lonk Cool - I've never made any inroads in the WP & Woo community TBH, found it quite cliquey and in self-denial with many woocommerc.com plugins we had to fork and fix in desperation from their lack of quality-control.

I've got a fair few core improvement recommendations that fell on deaf ears too - but I'm certain we could make it the operating-system for any organisation.

I guess what we're doing is kinda like all the Cloudron Apps but as WP plugins. It is opinionated and anti-microservices though, so not for everyone - yet.

If you have any contacts that want to get serious about this kind of ambition, I'm always available to talk with fellows that aren't afraid to shoot for the moon.

Lonkle

@jdaviescoates said in Changes to WordPress apps:

As I understand it (one of) the problem(s) is that I don't think it's currently possible to have multiple domains pointing to the same Cloudron app, and imho we'd need that to work for Multisite to be useful (so that all the subsites within the multisite can be mapped to their own TLD).
I guess that is what you mean by real domain aliases?

Yes, a real domain alias so that Wordpress sees it and routes it accordingly. It's a built in feature of Wordpress in Multisite. It's unbelievably useful.

marcusquinn

@jdaviescoates In the meantime, people can reproduce most of what multi-site does with these plugins for those wanting each site to stay contained and portable for resources:

I do intend to have another look at multi-site at some point as a way of quickly firing up demo instances, so I agree on the need, we just wanted to solve the same things in a more segregated way to keep options open.

Lonkle

@marcusquinn And that's the one area you and I diverge in which, I think, is good. You keep me thinking of the benefits of single site and I'll keep you thinking of the benefits of multisite.

marcusquinn

For interest, these aliases are all the same Cloudron app:

I'll ask the team for more details while we wait for an official option.

girish

I have pushed the fixes for the WordPress (Developer) app. See https://forum.cloudron.io/post/16775 . It now has LDAP support as well. New doc pages is at https://docs.cloudron.io/apps/wordpress-developer/

Lonkle

@girish said in Changes to WordPress apps:

I have pushed the fixes for the WordPress (Developer) app. See https://forum.cloudron.io/post/16775 . It now has LDAP support as well. New doc pages is at https://docs.cloudron.io/apps/wordpress-developer/

Can you activate LDAP post-installation or would I have to re-install? I broke Cloudron rn so I can't test an app to try and see if it's in the configuration settings.

girish

The SFTP issue is sorted out now. There is a config option in Services -> SFTP. By default, only admins can access files via SFTP. So, this is a breaking change in the next release.

jdaviescoates

@Lonk said in Changes to WordPress apps:

Can you activate LDAP post-installation or would I have to re-install?

Exactly what I'm thinking... I'll go and see...

imc67

@jdaviescoates I’ve updated one of my wp apps, installed the ldap plugin used the managed ldap settings and (just to be sure) after a restart of the app ldap works!

jdaviescoates

@imc67 said in Changes to WordPress apps:

installed the ldap plugin used the managed ldap settings

Aha, I also just updated and didn't see any LDAP support, but this is the step I'm missing!

@girish be nice if updating Unmanaged to the new Developer version auto-magically installed the LDAP plugin and settings!

jdaviescoates

@imc67 said in Changes to WordPress apps:

@jdaviescoates I’ve updated one of my wp apps, installed the ldap plugin used the managed ldap settings and (just to be sure) after a restart of the app ldap works!

When you go to Access Control in your updated app are you now seeing this? (as per how it looks if you install and choose LDAP on install)

Lonkle

@jdaviescoates said in Changes to WordPress apps:

@imc67 said in Changes to WordPress apps:

installed the ldap plugin used the managed ldap settings

Aha, I also just updated and didn't see any LDAP support, but this is the step I'm missing!

@girish be nice if updating Unmanaged to the new Developer version auto-magically installed the LDAP plugin and settings!

I would have missed that necessary step myself. The only reason I didn't check on my own install is because @girish made a "hot fix" for me for my development and I had a VM issue so I re-installed and was refusing to go through the setup because I was afraid cloudron-machine wouldn't work after setup. But @girish confirmed today it does work after setup so I'm going to have fun testing out this LDAP integration with WP and figure out how cloudron-machine works later.

girish

@jdaviescoates What @imc67 did should not be possible, so I am not sure how it works for him. The LDAP will only be available for new installations since this flag is chosen at install time and there is no way to change it post installation without tinkering with the database. Might be easier to export/import into a new install. Just backup current app, make a new LDAP based install and import that backup into new app.

imc67

@girish it was really as easy as the steps I’ve mentioned before.

girish

@imc67 Ah, "installed the ldap plugin used the managed ldap settings". I missed this. So you put the credentials of managed LDAP app into this existing unmanaged app? Note that this will stop working when the managed app goes away!

Lonkle

@girish said in Changes to WordPress apps:

@jdaviescoates What @imc67 did should not be possible, so I am not sure how it works for him. The LDAP will only be available for new installations since this flag is chosen at install time and there is no way to change it post installation without tinkering with the database. Might be easier to export/import into a new install. Just backup current app, make a new LDAP based install and import that backup into new app.

Oh, gotcha, it can only be set at install time. So, do you think your suggestion to "set the flag without DB voodoo" (which I'm gonna do because I can't get the cloudron app db to work with remote sql, I've tried a lot). Do you think that method will clash with multisite database URL location changes (Wordpress storing the URL in a different place than single site)?

imc67

@girish huh why? These “codes” / settings are even from a managed app on another Cloudron, aren’t they the same for all?

Lonkle

@girish Yes, also curious to understand what you mean by @imc67's hack will stop working...at some point (that I didn't understand "when the managed app goes away" when the managed app isn't going away)?

robi

@Lonk credentials are generated per app, he copied one app's LDAP credentials.

If that app goes away, so do the unique credentials.

imc67

@robi but there are no credentials in the settings of the LDAP plug-in at WP, only settings/code

(I tried to upload a 'scrolled' screenshot of the settings page but it's too big)

Schermafbeelding 2020-10-22 om 22.07.44.png

girish

@imc67 Ah yes, I see why it works.

From a security perspective, each app gets it's own addon credentials (database, redis, ldap etc). When app is installed/uninstalled, we create/destroy a separate username/password/database for each app. Now, Cloudron could have gone a step further and implemented a security measure that these credentials will work only when the specific app uses it. This can be done because each app has it's own IP address internally (via Docker). We haven't implemented this, and as a result, the credentials of one app (say mysql username/password/database) can be copied over to another app and it will work. But it will only work until the other app exists. When the other app is uninstalled or repaired/restored etc, the credentials are regenerated.

In the case of LDAP addon, there is a so called "bind" password which allows apps to make LDAP queries. We generate a bind password per app. However, currently, we don't enforce this password since some apps do not support it. This WordPress LDAP plugin we use is one such case (probably one of the remaining 3-4 apps in Cloudron). Because, it doesn't use a bind username/password, all you are copying over is the LDAP server credentials (server name/port which is the same across all cloudrons). So, this happens to work now. But later when we fix the plugin to use LDAP credentials, it will stop working.

Also, you will see inconsistency in the UI since Cloudron is not aware that LDAP is enabled for the app. You will see a different access control view than what @jdaviescoates posted. You also can't control which users have access to ldap from the Cloudron UI. In fact, I am going to guess only admins can access your WP install (since they are allowed by default).

imc67

@girish that's sad, as the expectations with "upgrading" the app to 'developer' suggested all the long awaited new functionality.

So what's the best, step-by-step approach?

Lonkle

@girish said in Changes to WordPress apps:

In the case of LDAP addon, there is a so called "bind" password which allows apps to make LDAP queries. We generate a bind password per app.

Where might I be able to find this bind password? I made up my own tiny PHP library for LDAP in the VPN Client and I did not use a bind password even though it's not required "for now". So, I would def like to fix that preemptively.

But later when we fix the plugin to use LDAP credentials, it will stop working.

Did you write this pugin yourself or do you need to make the fix upstream with another team?

girish

@imc67 said in Changes to WordPress apps:

@girish that's sad, as the expectations with "upgrading" the app to 'developer' suggested all the long awaited new functionality.

Yeah. For LDAP, we don't have a mechanism to easily to turn it on/off dynamically ie after an app installed. Let me discuss this with @nebulon to see if this is something we should do for Cloudron 6 because it's easy to do on Cloudron side (but we have to test with all the apps to check how well they cope).

If it's urgent, the easiest way is to just:

Backup current app. Download the backup config
Make a new install of WordPress (Developer), you can keep the existing app running.
Then import the backup config into this new app. App -> Backups -> Import. Upload the config from step 1.
Login to WP of the new app and install authLdap plugin. After doing so, Restart WordPress. Cloudron will configure the LDAP plugin on restart.
If all looks good, you can switch the location

girish

@Lonk said in Changes to WordPress apps:

Did you write this pugin yourself or do you need to make the fix upstream with another team?

It's authLdap. We have contributed patches in the past like this one. So, have to invest more time into adding bind password support.

Lonkle

@girish said in Changes to WordPress apps:

If it's urgent, the easiest way is to just:

Backup current app. Download the backup config
Make a new install of WordPress (Developer), you can keep the bet existing app running.
Then import the backup config into this new app. App -> Backups -> Import. Upload the config from step 1.
If all looks good, you can switch the location

That answered my question, this won't support multisite yet, but I can make some manual DB changes to still make this work and understand Cloudron better, win-win. ️

jdaviescoates

@girish said in Changes to WordPress apps:

Backup current app. Download the backup config
Make a new install of WordPress (Developer), you can keep the existing app running.
Then import the backup config into this new app. App -> Backups -> Import. Upload the config from step 1.
If all looks good, you can switch the location

Just did that. Worked a treat!

imc67

@girish said in Changes to WordPress apps:

Backup current app. Download the backup config
Make a new install of WordPress (Developer), you can keep the existing app running.
Then import the backup config into this new app. App -> Backups -> Import. Upload the config from step 1.
If all looks good, you can switch the location

just did that but the authLdap plugin is not installed after restoring the backup (a backup from before I manually added it).

girish

@imc67 said in Changes to WordPress apps:

just did that but the authLdap plugin is not installed after restoring the backup (a backup from before I manually added it).

You can just install it from WP Admin and restart WordPress (Cloudron will configure the plugin on restart). I will edit the instructions.

imc67

@girish that worked, I even saw already the settings in the ldap plugin but still restarted the app to be sure.

one thing: I expected a ldap logged in cloudron admin also should become admin in WP (in my previous but wrong setup this was the case)?

Lonkle

@imc67 said in Changes to WordPress apps:

@girish that worked, I even saw already the settings in the ldap plugin but still restarted the app to be sure.

one thing: I expected a ldap logged in cloudron admin also should become admin in WP (in my previous but wrong setup this was the case)?

So the LDAP plugin "syncs" the WP user database with Cloudron's then? I wonder if it sync both ways.

And also, what did all of your Cloudron users import as in Wordpress with the "official" non-hack-ed way of enabling LDAP?

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Changes to WordPress apps

Feature Parity

DESCRIPTION changes