Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved mysql addon fails to start

    Support
    update mysql
    3
    6
    414
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      dfldadm last edited by girish

      Hi,

      I'm running a cloudron 6.2.4 in which the disk went full (too many images of different versions in docker).

      Now the mysql addon (not the ubuntu mysql) doesn't come up again:

      2021-03-23T11:43:34.632721Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23-0ubuntu0.20.04.1) starting as process 11
      2021-03-23T11:43:34.646837Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
      2021-03-23T11:43:34.952073Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
      2021-03-23T11:43:34.959894Z 1 [ERROR] [MY-011092] [Server] Upgrading the data dictionary from dictionary version '0' is not supported.
      2021-03-23T11:43:34.960213Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
      2021-03-23T11:43:34.960380Z 0 [ERROR] [MY-010119] [Server] Aborting
      2021-03-23T11:43:35.479431Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.23-0ubuntu0.20.04.1)  (Ubuntu).
      

      Some people report this may happen if mysql is upgraded skipping versions but I cannot know whether this is the case/how cloudron handles this.

      Is there a way to recover from this? I cannot use my apps for this reason, not even stopping/starting or recovery is possible.

      Update: This is amazing. I encounter the exact same error on another instance in another hosting environment.

      Trying to fix this desperately I tried cloudron/mysql 2.3.2, 3.0.0, 3.0.3 and 3.0.6 (3.0.4 seems to be default) but nothing helped.

      girish 1 Reply Last reply Reply Quote 0
      • girish
        girish Staff @dfldadm last edited by girish

        @dfldadm Can you send us a mail to support@cloudron.io instead? It's hard to diagnose the problem from just the logs above.

        Also, it doesn't come up after freeing up some space is it? If you free up some space, all you have to do is systemctl restart box and it will re-initialize the addon as needed.

        1 Reply Last reply Reply Quote 0
        • girish
          girish Staff last edited by

          The root cause was that both servers ran out of disk space. So, the export of db from old version failed and thus couldn't import to new version.

          1 Reply Last reply Reply Quote 1
          • D
            dfldadm last edited by

            For the record, since the communication on this went private.

            The issue was solved by manual intervention.
            I have two takeaway messages:

            1

            If mysql doesn't come back up, it is possible to
            remove /home/yellowtent/platformdata/mysql (this is where the mysql addon stores it's data) and restart the box code using systemctl restart box.
            (Do this from outside the mysql container) This is apparently non-destructive.

            This fixed the first machine which hung up as it ran out of disk space (and this didn't change as space was freed up again)

            2

            • Do not upgrade box or any app while there's not plenty (5 Gig perhaps) of free space. It seems as if cloudron apps can get stuck in a bad state when space runs out while doing system things [DB migration, backups, updates] (I hope this is correct)
            • Monitor space closely, it's cheaper to buy more than to end up fixing a stuck setup...
            nebulon 1 Reply Last reply Reply Quote 1
            • nebulon
              nebulon Staff @dfldadm last edited by

              @dfldadm action 1) is only non-destricutive, if you can restore the apps afterwards! Anyways always ensure to have backups. We have had encountered various situations where a server runs out of disk space and to free up space, people deleted on-disk backups. So at least download those to a safe place beforehand.

              1 Reply Last reply Reply Quote 1
              • girish
                girish Staff last edited by

                Generally, I am wary of putting this information for general use (and even in docs) because many things can go wrong . So, please, if you care about the data, don't hesitate to contact us on support@ . We help regardless of the subscription plan.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Powered by NodeBB