Website running on managed WP unreachable after latest automatic upgrade
-
This past night one of my wordpress websites, running the managed version, was updated automatically from v2.21.3 to v2.22.0 and now it isn't reachable anymore.
According to the cloudron dashboard it is running, but in the logs I noticed the following error which is being repeated continuously:
Mar 15 08:33:28 ==> Changing permissions Mar 15 08:33:29 => Updating db settings Mar 15 08:33:29 Success: Updated the constant 'DB_HOST' in the 'wp-config.php' file with the value 'mysql:3306'. Mar 15 08:33:29 Success: Updated the constant 'DB_NAME' in the 'wp-config.php' file with the value '<redacted>'. Mar 15 08:33:29 Success: Updated the constant 'DB_USER' in the 'wp-config.php' file with the value '<redacted>'. Mar 15 08:33:30 Success: Updated the constant 'DB_PASSWORD' in the 'wp-config.php' file with the value '<redacted>'. Mar 15 08:33:30 ==> Updating wordpress database Mar 15 08:33:31 Success: WordPress database already at latest db version 51917. Mar 15 08:33:31 ==> Updating domain related settings Mar 15 08:33:31 Success: Updated the constant 'WP_HOME' in the 'wp-config.php' file with the value 'https://subdomain.domain.tld'. Mar 15 08:33:31 Success: Updated the constant 'WP_SITEURL' in the 'wp-config.php' file with the value 'https://subdomain.domain.tld'. Mar 15 08:33:32 Success: Value passed for 'siteurl' option is unchanged. Mar 15 08:33:33 Success: Value passed for 'home' option is unchanged. Mar 15 08:33:33 ==> Configuring smtp mail Mar 15 08:33:33 Error: Could not get 'wp_mail_smtp' option. Does it exist? Mar 15 08:33:35 Success: Value passed for 'smtp_mailer_options' option is unchanged. Mar 15 08:33:35 ==> Configuring LDAP Mar 15 08:33:37 Success: Value passed for 'authLDAPOptions' option is unchanged. Mar 15 08:33:37 ==> Migrating database tables to have a prefix Mar 15 08:33:37 ==> Renaming using RENAME TABLE `<redacted>`.`banhammer` TO `<redacted>`.`wp_banhammer`,`<redacted>`.`commentmeta` TO `<redacted>`.`wp_commentmeta`,`<redacted>`.`comments` TO `<redacted>`.`wp_comments`,`<redacted>`.`frmt_form_entry` TO `<redacted>`.`wp_frmt_form_entry`,`<redacted>`.`frmt_form_entry_meta` TO `<redacted>`.`wp_frmt_form_entry_meta`,`<redacted>`.`frmt_form_views` TO `<redacted>`.`wp_frmt_form_views`,`<redacted>`.`icwp_wpsf_audit_trail` TO `<redacted>`.`wp_icwp_wpsf_audit_trail`,`<redacted>`.`icwp_wpsf_events` TO `<redacted>`.`wp_icwp_wpsf_events`,`<redacted>`.`icwp_wpsf_geoip` TO `<redacted>`.`wp_icwp_wpsf_geoip`,`<redacted>`.`icwp_wpsf_ip_lists` TO `<redacted>`.`wp_icwp_wpsf_ip_lists`,`<redacted>`.`icwp_wpsf_notes` TO `<redacted>`.`wp_icwp_wpsf_notes`,`<redacted>`.`icwp_wpsf_reports` TO `<redacted>`.`wp_icwp_wpsf_reports`,`<redacted>`.`icwp_wpsf_sessions` TO `<redacted>`.`w Mar 15 08:33:37 ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '`w' at line 1 Mar 15 08:34:38 ==> Changing permissions ...So it seems that after the update an incomplete query is being executed for prefixing the database tables?
-
Seems like there is a bug in the table migration then. Please restore to the previous version to get up and running again and then maybe clone the app to test the update in an isolated way without interfering with your production website. Afterwards you can enable remote SSH for us and send us a mail with your dashboard domain/IP to support@cloudron.io for investigation.
-
@nebulon Fortunately it's not a production website and therefore I didn't restore it yet to a previous version in order to be able to provide more details in case you need them.
I noticed @girish said in WordPress Managed - Package updates:
[2.22.0]
- Change the table prefix to
wp_. This makes it easier to migrate to the Developer package and is also better supported by plugins.
So it's probably not a coincidence that this is happening and maybe other (production) installations might be affected by this too.
- Change the table prefix to
-
-
@guyds I have pushed a new package with a fix now. Can you please try the new one? i.e try updating from v2.21.3 to v2.22.0-1 (and not from 2.22.0). If that doesn't work too, can you please write to us at support@cloudron.io ? I would like to understand what's going wrong.
-
@girish I see, but if it's marked as unstable then why is it still installed via an automatic update? Shouldn't unstable updates be ignored by the automatic update process?
I'll try to roll back to an earlier backup and then upgrade to v2.22.0-1 and let you know the results.
-
@girish I successfully downgraded to v2.21.3 but the upgrade to v2.22.0-1 still fails with a similar, but slightly different error:
Mar 15 20:16:55 ==> Changing permissions Mar 15 20:16:55 => Updating db settings Mar 15 20:16:56 Success: Updated the constant 'DB_HOST' in the 'wp-config.php' file with the value 'mysql:3306'. Mar 15 20:16:56 Success: Updated the constant 'DB_NAME' in the 'wp-config.php' file with the value '<redacted>'. Mar 15 20:16:56 Success: Updated the constant 'DB_USER' in the 'wp-config.php' file with the value '<redacted>'. Mar 15 20:16:56 Success: Updated the constant 'DB_PASSWORD' in the 'wp-config.php' file with the value '<redacted>'. Mar 15 20:16:57 ==> Updating wordpress database Mar 15 20:16:57 Success: WordPress database already at latest db version 51917. Mar 15 20:16:57 ==> Updating domain related settings Mar 15 20:16:58 Success: Updated the constant 'WP_HOME' in the 'wp-config.php' file with the value 'https://subdomain.domain.tld'. Mar 15 20:16:58 Success: Updated the constant 'WP_SITEURL' in the 'wp-config.php' file with the value 'https://subdomain.domain.tld'. Mar 15 20:16:59 Success: Value passed for 'siteurl' option is unchanged. Mar 15 20:17:00 Success: Value passed for 'home' option is unchanged. Mar 15 20:17:00 ==> Configuring smtp mail Mar 15 20:17:01 Error: Could not get 'wp_mail_smtp' option. Does it exist? Mar 15 20:17:03 Success: Value passed for 'smtp_mailer_options' option is unchanged. Mar 15 20:17:03 ==> Configuring LDAP Mar 15 20:17:04 Success: Value passed for 'authLDAPOptions' option is unchanged. Mar 15 20:17:05 ==> Migrating database tables to have a prefix Mar 15 20:17:05 ==> Renaming using RENAME TABLE `banhammer` TO `wp_banhammer`,`commentmeta` TO `wp_commentmeta`,`comments` TO `wp_comments`,`frmt_form_entry` TO `wp_frmt_form_entry`,`frmt_form_entry_meta` TO `wp_frmt_form_entry_meta`,`frmt_form_views` TO `wp_frmt_form_views`,`icwp_wpsf_audit_trail` TO `wp_icwp_wpsf_audit_trail`,`icwp_wpsf_events` TO `wp_icwp_wpsf_events`,`icwp_wpsf_geoip` TO `wp_icwp_wpsf_geoip`,`icwp_wpsf_ip_lists` TO `wp_icwp_wpsf_ip_lists`,`icwp_wpsf_notes` TO `wp_icwp_wpsf_notes`,`icwp_wpsf_reports` TO `wp_icwp_wpsf_reports`,`icwp_wpsf_sessions` TO `wp_icwp_wpsf_sessions`,`icwp_wpsf_traffic` TO `wp_icwp_wpsf_traffic`,`itsec_bans` TO `wp_itsec_bans`,`itsec_distributed_storage` TO `wp_itsec_distributed_storage`,`itsec_fingerprints` TO `wp_itsec_fingerprints`,`itsec_geolocation_cache` TO `wp_itsec_geolocation_cache`,`itsec_lockouts` TO `wp_itsec_lockouts`,`itsec_logs` TO `wp_itsec_logs`,`itsec_mutexes` TO `wp_itsec_mutexes`,`itsec_opaque_tokens` TO `wp_itsec_opaque_tokens`,`itsec_temp` TO `wp_itsec_temp`,`itsec_user_groups` TO `wp_itsec_u Mar 15 20:17:05 ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '`wp_itsec_u' at line 1Apparently it's still hitting a length limit...
-
-
@guyds thanks for testing! Do you think you can send me the log file to support@cloudron.io ? (I really just want the full RENAME line which is getting truncated here. Maybe you can just paste that fully here?) You can do this by Downloading the log file and getting that line.
-
@girish said in Website running on managed WP unreachable after latest automatic upgrade:
@guyds When we make a thing is unstable, our managed instances and internal instances are updated first. It then starts rolling out things slowly. I think your app was one of the super early adopters.
Ok, that's a plausible explanation.
@girish said in Website running on managed WP unreachable after latest automatic upgrade:
@guyds thanks for testing! Do you think you can send me the log file to support@cloudron.io ? (I really just want the full RENAME line which is getting truncated here. Maybe you can just paste that fully here?) You can do this by Downloading the log file and getting that line.
I downloaded the full logfile but even then the RENAME command is truncated:
2022-03-15T19:35:24.000Z ==> Migrating database tables to have a prefix 2022-03-15T19:35:24.000Z ==> Renaming using RENAME TABLE `banhammer` TO `wp_banhammer`,`commentmeta` TO `wp_commentmeta`,`comments` TO `wp_comments`,`frmt_form_entry` TO `wp_frmt_form_entry`,`frmt_form_entry_meta` TO `wp_frmt_form_entry_meta`,`frmt_form_views` TO `wp_frmt_form_views`,`icwp_wpsf_audit_trail` TO `wp_icwp_wpsf_audit_trail`,`icwp_wpsf_events` TO `wp_icwp_wpsf_events`,`icwp_wpsf_geoip` TO `wp_icwp_wpsf_geoip`,`icwp_wpsf_ip_lists` TO `wp_icwp_wpsf_ip_lists`,`icwp_wpsf_notes` TO `wp_icwp_wpsf_notes`,`icwp_wpsf_reports` TO `wp_icwp_wpsf_reports`,`icwp_wpsf_sessions` TO `wp_icwp_wpsf_sessions`,`icwp_wpsf_traffic` TO `wp_icwp_wpsf_traffic`,`itsec_bans` TO `wp_itsec_bans`,`itsec_distributed_storage` TO `wp_itsec_distributed_storage`,`itsec_fingerprints` TO `wp_itsec_fingerprints`,`itsec_geolocation_cache` TO `wp_itsec_geolocation_cache`,`itsec_lockouts` TO `wp_itsec_lockouts`,`itsec_logs` TO `wp_itsec_logs`,`itsec_mutexes` TO `wp_itsec_mutexes`,`itsec_opaque_tokens` TO `wp_itsec_opaque_tokens`,`itsec_temp` TO `wp_itsec_temp`,`itsec_user_groups` TO `wp_itsec_u 2022-03-15T19:35:24.000Z ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '`wp_itsec_u' at line 1I created a support ticket.
-
@guyds thanks! I sent you a reply from support. But from your mail it seems it is not related to SQL query length as I initially thought but some bash scripting issue.
-
The issue here was the GROUP_CONCAT has a default length of 1024 or so. Setting session variable
SET group_concat_max_len=10000fixes the issue. -
-
@jeau Can you check if it's updating to v2.22.0-5 ? If you can clone from latest backup and update, I am happy to look into this if you can write to support@cloudron.io.
-
@jeau mm, per https://docs.civicrm.org/installation/en/latest/wordpress/, CiviCRM needs to be told about the prefix as well.
It's not entirely clear but it looks like we have to set
$db_prefix = 'wp_';in settings.php -
It seems civicrm does not use WP's
table_prefixat all. Well, I pushed a new package which simply ignore civicrm tables. -
@girish Ignoring the civicrm tables works.
However, I have the same kind of problem with an integration of MRBS in Wordpress for authentication. As the tables were properly renamed during the update, I simply set the ad hoc variable (
$db_tbl_prefix = "wp_mrbs_";) in the configuration file and it works.But I guess there are other use cases where renaming tables causes problems.
-
@jeau Good to know, thanks!
We have to do this migration since WordPress it says it doesn't support not having a table prefix. See:
Let me see how we can work around this.