Vaultwarden fails to start after update – DB migration error (SSO)

[sangemaru]

Description

Aplication not responding. Investigated logs show database migration error.

Steps to reproduce

Vaultwarden App updated to 1.23.0

Logs

2025-12-30T08:56:20Z /--------------------------------------------------------------------\
2025-12-30T08:56:20Z 0: vaultwarden::init_logging::{{closure}}
2025-12-30T08:56:20Z 10: std::sys::backtrace::__rust_begin_short_backtrace
2025-12-30T08:56:20Z 11: main
2025-12-30T08:56:20Z 12: <unknown>
2025-12-30T08:56:20Z 13: __libc_start_main
2025-12-30T08:56:20Z 14: _start
2025-12-30T08:56:20Z 1: std::panicking::panic_with_hook
2025-12-30T08:56:20Z 2: std::panicking::panic_handler::{{closure}}
2025-12-30T08:56:20Z 3: std::sys::backtrace::__rust_end_short_backtrace
2025-12-30T08:56:20Z 4: __rustc::rust_begin_unwind
2025-12-30T08:56:20Z 5: core::panicking::panic_fmt
2025-12-30T08:56:20Z 6: core::result::unwrap_failed
2025-12-30T08:56:20Z 7: vaultwarden::db::DbPool::from_config
2025-12-30T08:56:20Z 8: vaultwarden::main::{{closure}}
2025-12-30T08:56:20Z 9: vaultwarden::main
2025-12-30T08:56:20Z => Exporting env vars expected by Vaultwarden
2025-12-30T08:56:20Z => Starting Bitwarden
2025-12-30T08:56:20Z Please generate a secure Argon2 PHC string by using `vaultwarden hash` or `argon2`.
2025-12-30T08:56:20Z See: https://github.com/dani-garcia/vaultwarden/wiki/Enabling-admin-page#secure-the-admin_token
2025-12-30T08:56:20Z [2025-12-30 08:56:20.876][panic][ERROR] thread 'main' panicked at 'Error running migrations: QueryError(DieselMigrationName { name: "2024-03-06-170000_add_sso_users", version: MigrationVersion("20240306170000") }, DatabaseError(Unknown, "Referencing column 'user_uuid' and referenced column 'uuid' in foreign key constraint 'sso_users_ibfk_1' are incompatible."))': src/db/mod.rs:505

Troubleshooting Already Performed

Restored 1.22.2 application Backup restored functionality

System Details

Generate Diagnostics Data

Diagnostic Data sent to support@cloudron.io

Cloudron Version

9.0.15

Ubuntu Version

Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.3 LTS
Release:	24.04
Codename:	noble

Cloudron installation method

• Manual with ./cloudron-setup

Output of cloudron-support --troubleshoot

Vendor: netcup Product: KVM Server
Linux: 6.8.0-90-generic
Ubuntu: noble 24.04
Execution environment: kvm
Processor: AMD EPYC 9645 96-Core Processor
BIOS pc-i440fx-9.2  CPU @ 2.0GHz x 8
RAM: 16370004KB
Disk: /dev/vda1       425G
[OK]	node version is correct
[OK]	IPv6 is enabled and public IPv6 address is working
[OK]	docker is running
[OK]	docker version is correct
[OK]	MySQL is running
[OK]	nginx is running
[OK]	dashboard cert is valid
[OK]	dashboard is reachable via loopback
[OK]	No pending database migrations
[OK]	Service 'mysql' is running and healthy
[OK]	Service 'postgresql' is running and healthy
[OK]	Service 'mongodb' is running and healthy
[OK]	Service 'mail' is running and healthy
[OK]	Service 'graphite' is running and healthy
[OK]	Service 'sftp' is running and healthy
[OK]	box v9.0.15 is running
[OK]	netplan is good
[OK]	DNS is resolving via systemd-resolved
[OK]	Dashboard is reachable via domain name
[WARN]	Domain serenichron.agency expiry check skipped because whois does not have this information
[OK]	unbound is running

[fbartels]

This has been reported along with a potential fix in https://forum.cloudron.io/topic/14812/vaultwarden-fails-to-start-after-update-db-migration-error-sso

[jadudm]

The fix that @inibudi posted worked for me.

[jdaviescoates]

@archos said in Vaultwarden fails to start after update – DB migration error (SSO):

After updating the Vaultwarden app on Cloudron

From/ to which version?

[james]

Hello @jdaviescoates
From version Cloudron package version 1.22.2 => Vaultwarden Version 1.34.3 to Cloudron package version 1.23.0 => Vaultwarden Version 1.35.0

[jdaviescoates]

@james OK, thanks. Looks like that is the pending update I have. Is there anyway to avoid having to do the fix above post update? Might another Cloudron package be released that has the fix?

[mazarian]

Thank you! This saved me a lot of time and fixed the issue. I followed your steps and appreciated the ChatGPT workaround.

[james]

Hello @jdaviescoates

@jdaviescoates said in Vaultwarden fails to start after update – DB migration error (SSO):

Might another Cloudron package be released that has the fix?

A fresh Vaultwarden installation will not have this issue.
About including a fix within the package.
We will have to look into it.

[james]

Hello @vaultwarden users

Since users have voiced their concerns about manually editing the database I have created a bash script that does it for you.
Please follow these steps if you have this issue:

1. create an app backup of your @vaultwarden Cloudron app
2. put the erroring @vaultwarden app in recovery mode
3. open the File Manager of your @vaultwarden app and create a fix_db.sh file
4. copy and paste the following script into the just created fix_db.sh file:

   #!/bin/bash
   
   echo "=> Checking if DEFAULT_CHARACTER_SET_NAME and DEFAULT_COLLATION_NAME are set to utf8mb4 and utf8mb4_unicode_ci"
   
   CURRENT_DEFAULT_CHARACTER_SET_NAME=$(mysql --silent --skip-column-names --user=${CLOUDRON_MYSQL_USERNAME} --password=${CLOUDRON_MYSQL_PASSWORD} --host=${CLOUDRON_MYSQL_HOST} ${CLOUDRON_MYSQL_DATABASE} -e "SELECT DEFAULT_CHARACTER_SET_NAME FROM information_schema.SCHEMATA WHERE SCHEMA_NAME = '${CLOUDRON_MYSQL_DATABASE}';" 2>/dev/null)
   CURRENT_DEFAULT_COLLATION_NAME=$(mysql --silent --skip-column-names --user=${CLOUDRON_MYSQL_USERNAME} --password=${CLOUDRON_MYSQL_PASSWORD} --host=${CLOUDRON_MYSQL_HOST} ${CLOUDRON_MYSQL_DATABASE} -e "SELECT DEFAULT_COLLATION_NAME FROM information_schema.SCHEMATA WHERE SCHEMA_NAME = '${CLOUDRON_MYSQL_DATABASE}';" 2>/dev/null)
   FIX_NEEDED=0
   
   if [ "${CURRENT_DEFAULT_CHARACTER_SET_NAME}" != "utf8mb4" ] || [ "${CURRENT_DEFAULT_COLLATION_NAME}" != "utf8mb4_unicode_ci" ]; then
       echo "=> DEFAULT_CHARACTER_SET_NAME or DEFAULT_COLLATION_NAME is not set to utf8mb4 or utf8mb4_unicode_ci."
       echo "==> Current values are: DEFAULT_CHARACTER_SET_NAME=${CURRENT_DEFAULT_CHARACTER_SET_NAME}, DEFAULT_COLLATION_NAME=${CURRENT_DEFAULT_COLLATION_NAME}"
       FIX_NEEDED=1
   else
       echo "=> DEFAULT_CHARACTER_SET_NAME is ${CURRENT_DEFAULT_CHARACTER_SET_NAME} and DEFAULT_COLLATION_NAME is ${CURRENT_DEFAULT_COLLATION_NAME}"
       echo "=> Nothing to do - quitting."
       FIX_NEEDED=0
       exit 0
   fi
   
   if [ ${FIX_NEEDED} -eq 1 ]; then
       read -p "Have you created a backup of your Cloudron Vaultwarden app? (y/N): " yn
       case $yn in
           [Yy]* )
               echo "=> Setting database character set and collation to utf8mb4 and utf8mb4_unicode_ci."
               mysql --user=${CLOUDRON_MYSQL_USERNAME} --password=${CLOUDRON_MYSQL_PASSWORD} --host=${CLOUDRON_MYSQL_HOST} ${CLOUDRON_MYSQL_DATABASE} -e "ALTER DATABASE \`${CLOUDRON_MYSQL_DATABASE}\` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;" 2>/dev/null
               echo "=> Converting all tables to character set utf8mb4 and collation utf8mb4_unicode_ci."
               mysql --silent --skip-column-names \
     --user="${CLOUDRON_MYSQL_USERNAME}" \
     --password="${CLOUDRON_MYSQL_PASSWORD}" \
     --host="${CLOUDRON_MYSQL_HOST}" \
     "${CLOUDRON_MYSQL_DATABASE}" \
     -e "SELECT CONCAT('ALTER TABLE \`', TABLE_NAME,'\` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;') AS CharSetConvert FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='${CLOUDRON_MYSQL_DATABASE}' AND TABLE_TYPE=\"BASE TABLE\";" 2>/dev/null | while read -r sql_command; do
                   echo "==> Executing: ${sql_command}"
                   mysql --user=${CLOUDRON_MYSQL_USERNAME} --password=${CLOUDRON_MYSQL_PASSWORD} --host=${CLOUDRON_MYSQL_HOST} ${CLOUDRON_MYSQL_DATABASE} -e "SET foreign_key_checks=0; ${sql_command} SET foreign_key_checks=1;" 2>/dev/null
               done
               ;;
           [Nn]* )
               echo "=> Please create a backup of your database before proceeding with the fix. Exiting."
               exit 1
               ;;
           * )
               echo "=> Invalid response. Please answer with y or n. Exiting."
               exit 1
               ;;
       esac
   fi
   

5. open the Web Terminal of your @vaultwarden Cloudron app
6. execute the following command:

   bash /app/data/fix_db.sh
   

7. Execute the following command to start @vaultwarden:

   /app/pkg/start.sh
   

8. Validate if your @vaultwarden is now working correctly
9. If validated working, you can delete the fix_db.sh file and disable the recovery mode of your @vaultwarden Cloudron app

---

If you run into any issues, copy the output of the terminal, post it here and restore your app from the backup created.

[ekevu123]

I confirm this works!

[andreasdueren]

@james this won’t fix the issue of logging everyone out though I assume?

Edit: just saw the newest update.

[sangemaru]

@james I also confirm this worked.

[murgero]

Confirming this fix worked however the latest package version still doesn't fix this problem.

[jdaviescoates]

@james said in Vaultwarden fails to start after update – DB migration error (SSO):

From version Cloudron package version 1.22.2 => Vaultwarden Version 1.34.3 to Cloudron package version 1.23.0 => Vaultwarden Version 1.35.0

Despite my worries and concerns, I never actually hit this problem. Guess I was lucky. But I wonder why?

My Vaultwarden started fine after doing that update. And then I immediately updated to the next version without issue too.

[Grienauer]

@james Hi thx. But how to use the web terminal if it is not starting/responding? At least this is the Problem on my side currently. So I can not execute any commands there…

[andreasdueren]

@Grienauer You need to put it into recovery mode, execure the script, take it out of it again

[Grienauer]

Yes, this did the trick. Thank you.
Crazy superfast response from you btw. thx!

[factord]

@james it doesnt work for me, here is the output

root@b2a9826b-7631-435b-91a0-d0a505476ee9:/app/code# bash /app/data/fix_db.sh
/app/data/fix_db.sh: line 2: $'\r': command not found
=> Checking if DEFAULT_CHARACTER_SET_NAME and DEFAULT_COLLATION_NAME are set to utf8mb4 and utf8mb4_unicode_ci
/app/data/fix_db.sh: line 4: $'\r': command not found
/app/data/fix_db.sh: line 8: $'\r': command not found
/app/data/fix_db.sh: line 22: syntax error near unexpected token `$'in\r''
'app/data/fix_db.sh: line 22: `    case $yn in
root@b2a9826b-7631-435b-91a0-d0a505476ee9:/app/code# /app/pkg/start.sh
=> Exporting env vars expected by Vaultwarden
=> Starting Bitwarden
/--------------------------------------------------------------------\
|                        Starting Vaultwarden                        |
|--------------------------------------------------------------------|
| This is an *unofficial* Bitwarden implementation, DO NOT use the   |
| official channels to report bugs/features, regardless of client.   |
| Send usage/configuration questions or feature requests to:         |
|   https://github.com/dani-garcia/vaultwarden/discussions or        |
|   https://vaultwarden.discourse.group/                             |
| Report suspected bugs/issues in the software itself at:            |
|   https://github.com/dani-garcia/vaultwarden/issues/new            |
\--------------------------------------------------------------------/

[INFO] Using saved config from `/app/data/config.json` for configuration.

[2026-01-03 17:24:33.038][panic][ERROR] thread 'main' panicked at 'Error running migrations: QueryError(DieselMigrationName { name: "2024-03-06-170000_add_sso_users", version: MigrationVersion("20240306170000") }, DatabaseError(Unknown, "Referencing column 'user_uuid' and referenced column 'uuid' in foreign key constraint 'sso_users_ibfk_1' are incompatible."))': src/db/mod.rs:505
   0: vaultwarden::init_logging::{{closure}}
   1: std::panicking::panic_with_hook
   2: std::panicking::panic_handler::{{closure}}
   3: std::sys::backtrace::__rust_end_short_backtrace
   4: __rustc::rust_begin_unwind
   5: core::panicking::panic_fmt
   6: core::result::unwrap_failed
   7: vaultwarden::db::DbPool::from_config
   8: vaultwarden::main::{{closure}}
   9: vaultwarden::main
  10: std::sys::backtrace::__rust_begin_short_backtrace
  11: main
  12: <unknown>
  13: __libc_start_main
  14: _start

root@b2a9826b-7631-435b-91a0-d0a505476ee9:/app/code# 

PS: i restored to 1.22.2 to get it working for now.

[james]

Hello @factord
This looks like a copy-paste issue from Windows.
\r in line 2 is a blank line, so a line break.
When you copy and pasted, it pasted the line breaks windows style.

Please describe how you have copy and pasted the code into the /app/data/fix_db.sh file.

[ChristopherMag]

I have the same issue as @factord I am on windows 11 and created the file through the file manager, pasted the contents copied from the post into the file using the built in file editor that is a part of the file manager, saved the file, opened the web terminal and ran the bash /app/data/fix_db.sh command.