Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. 6.3.3 a few quirks

6.3.3 a few quirks

Scheduled Pinned Locked Moved Solved Support
update
27 Posts 8 Posters 3.0k Views 9 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • luckowL luckow

      @girish easy. skip backup 🙂 There is a checkbox.

      girishG Do not disturb
      girishG Do not disturb
      girish
      Staff
      wrote on last edited by
      #9

      @luckow Thanks for access to the server, I found the issue but not sure why it happens yet. The box code temporarily lost connection to the database and caused some state to be lost. Workaround: I did a "systemctl restart box" and it picked up from where it left off.

      Thanks for reporting! Let's see how common this is.

      2021-06-25T12:07:36.791Z box:apps BoxError: connect ETIMEDOUT
          at /home/yellowtent/box/src/settingsdb.js:26:36
          at Query.queryCallback [as _callback] (/home/yellowtent/box/src/database.js:96:42)
          at Query.Sequence.end (/home/yellowtent/box/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
          at /home/yellowtent/box/node_modules/mysql/lib/Pool.js:205:13
          at Handshake.onConnect (/home/yellowtent/box/node_modules/mysql/lib/Pool.js:58:9)
          at Handshake.<anonymous> (/home/yellowtent/box/node_modules/mysql/lib/Connection.js:526:10)
          at Handshake._callback (/home/yellowtent/box/node_modules/mysql/lib/Connection.js:488:16)
          at Handshake.Sequence.end (/home/yellowtent/box/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
          at Protocol.handleNetworkError (/home/yellowtent/box/node_modules/mysql/lib/protocol/Protocol.js:369:14)
          at PoolConnection.Connection._handleNetworkError (/home/yellowtent/box/node_modules/mysql/lib/Connection.js:418:18)
      
      1 Reply Last reply
      1
      • robiR Offline
        robiR Offline
        robi
        wrote on last edited by robi
        #10

        I have a similar issue where 55 apps were backed up, but then many had to reconfigure so that took 20m, and all of the apps redirect to the "you found a cloudron in the wild" page.

        Restarted Nginx from services menu, no change.
        Restarted box from CLI, no change. (It seems we could also use a restart button for the cloudron service.)
        All my apps are offline.

        Update: After 30m or so, apps are coming back online.

        Conscious tech

        1 Reply Last reply
        1
        • girishG Do not disturb
          girishG Do not disturb
          girish
          Staff
          wrote on last edited by
          #11

          The update has to do a "rebuild" of all the containers and services because of the multi-host changes. This can take a bit depending on how fast the server CPU and disk is. It's best to follow the progress under /home/yellowtent/platformdata/logs/box.log. That said 30m does seem quite excessive. Maybe the databases are taking a long time to come up? If you can see in the logs where the most time is spent, would be good to know.

          d19dotcaD 1 Reply Last reply
          0
          • girishG girish

            The update has to do a "rebuild" of all the containers and services because of the multi-host changes. This can take a bit depending on how fast the server CPU and disk is. It's best to follow the progress under /home/yellowtent/platformdata/logs/box.log. That said 30m does seem quite excessive. Maybe the databases are taking a long time to come up? If you can see in the logs where the most time is spent, would be good to know.

            d19dotcaD Offline
            d19dotcaD Offline
            d19dotca
            wrote on last edited by d19dotca
            #12

            @girish Hi Girish, I think I had mentioned this a long time ago, but for what it's worth... I think it'd be helpful to include in the release notes a bit of an "info/alert" when there is to be more downtime than usual.

            For me, it's alarming and jarring to see a random lengthy update when the usual update time to Cloudron is only a minute or two. So when it unexpectedly takes 10-30+ minutes (depending on performance of the server I guess) due to the containers needing to be re-created for example, it's not the best user experience.

            I tend to make "30 minute" maintenance windows to sort of cover myself (and during off-peak hours of course) so the length of downtime isn't necessarily the main issue, but for me at least I was caught by surprise at the length it was taking and started to panic a little thinking something was wrong because it's so unusual for it to take that long. Thankfully I saw the containers being recreated when I looked at the system logs so knew it was okay and just needed some time, but I think having that pre-warning would be very helpful so admins at least know to expect a lengthier update instead of the usual update time.

            --
            Dustin Dauncey
            www.d19.ca

            1 Reply Last reply
            3
            • robiR Offline
              robiR Offline
              robi
              wrote on last edited by
              #13

              do these "rebuild" changes need to be a blocking operation?

              can it be redone in a more gentle, async way and minimize overall downtime?

              this will only amplify as we add more Cloudrons into the mix.

              Conscious tech

              1 Reply Last reply
              0
              • ? Offline
                ? Offline
                A Former User
                wrote on last edited by A Former User
                #14

                Having the same issue.
                Screen Shot 2021-06-27 at 10.44.12 PM.png

                Is this what others experienced?

                Screen Shot 2021-06-27 at 10.47.16 PM.png

                @girish This is my nginx logs... I don't have physical access to the server right now but I can drive over to take a look if you have ideas.

                1 Reply Last reply
                0
                • ? Offline
                  ? Offline
                  A Former User
                  wrote on last edited by A Former User
                  #15

                  Ok so now after a restart it looks like nginx just refuses to start...

                  Fuck... guess im nuking my install, restoring from wasabi, and praying to the dieties i dont believe in that it just goes back to normal.

                  @staff any idea what might have gone wrong? Can I set up a time with one of you to do a supervised update?

                  Also I guess the contents of my volumes are fucked as well...

                  girishG 1 Reply Last reply
                  0
                  • ? A Former User

                    Ok so now after a restart it looks like nginx just refuses to start...

                    Fuck... guess im nuking my install, restoring from wasabi, and praying to the dieties i dont believe in that it just goes back to normal.

                    @staff any idea what might have gone wrong? Can I set up a time with one of you to do a supervised update?

                    Also I guess the contents of my volumes are fucked as well...

                    girishG Do not disturb
                    girishG Do not disturb
                    girish
                    Staff
                    wrote on last edited by
                    #16

                    @atridad That's not good, do you think I can have access to your server? If you enable SSH access, I can look into first thing tomorrow. thanks and sorry for the issue!

                    ? 1 Reply Last reply
                    0
                    • ? Offline
                      ? Offline
                      A Former User
                      wrote on last edited by
                      #17

                      PXL_20210628_055812487~2.jpg

                      This seems to be the error...

                      1 Reply Last reply
                      0
                      • girishG girish

                        @atridad That's not good, do you think I can have access to your server? If you enable SSH access, I can look into first thing tomorrow. thanks and sorry for the issue!

                        ? Offline
                        ? Offline
                        A Former User
                        wrote on last edited by A Former User
                        #18

                        @girish Sure. I'll DM you

                        EDIT: That works. Only thing is I cant access the dashboard which makes it hard to enable ssh access. I can make you an account if you need? I'm going to be up for the next 2 hours so, so let me know and I can set things up.

                        M 1 Reply Last reply
                        0
                        • ? A Former User

                          @girish Sure. I'll DM you

                          EDIT: That works. Only thing is I cant access the dashboard which makes it hard to enable ssh access. I can make you an account if you need? I'm going to be up for the next 2 hours so, so let me know and I can set things up.

                          M Online
                          M Online
                          msbt
                          App Dev
                          wrote on last edited by
                          #19

                          @atridad you could run cloudron-support --enable-ssh - this should have the same effect

                          ? 1 Reply Last reply
                          1
                          • M msbt

                            @atridad you could run cloudron-support --enable-ssh - this should have the same effect

                            ? Offline
                            ? Offline
                            A Former User
                            wrote on last edited by
                            #20

                            @msbt said in 6.3.3 a few quirks:

                            cloudron-support --enable-ssh

                            I love you.

                            Seriously though thanks. I was pacing around trying to figure out how to do this lol

                            1 Reply Last reply
                            0
                            • nebulonN Away
                              nebulonN Away
                              nebulon
                              Staff
                              wrote on last edited by
                              #21

                              Ok at least for @atridad the issue was that for some reason old dashboard nginx configs were still around and referencing old SSL certificates which were now purged from the system. This made nginx fail to startup, causing the rest of the side-effects.

                              necrevistonnezrN 1 Reply Last reply
                              0
                              • girishG Do not disturb
                                girishG Do not disturb
                                girish
                                Staff
                                wrote on last edited by
                                #22

                                Ah thanks, I had fixed this yesterday - https://git.cloudron.io/cloudron/box/-/commit/c052882de9c93ed00949514e234ade813a465598 . It will be in 6.3.4

                                1 Reply Last reply
                                0
                                • nebulonN nebulon

                                  Ok at least for @atridad the issue was that for some reason old dashboard nginx configs were still around and referencing old SSL certificates which were now purged from the system. This made nginx fail to startup, causing the rest of the side-effects.

                                  necrevistonnezrN Offline
                                  necrevistonnezrN Offline
                                  necrevistonnezr
                                  wrote on last edited by
                                  #23

                                  @nebulon said in 6.3.3 a few quirks:

                                  Ok at least for @atridad the issue was that for some reason old dashboard nginx configs were still around and referencing old SSL certificates which were now purged from the system. This made nginx fail to startup, causing the rest of the side-effects.

                                  That had been my problem as well (more or less): https://forum.cloudron.io/topic/5263/cloudron-does-not-start-after-update-to-6-3-3

                                  ? 1 Reply Last reply
                                  1
                                  • necrevistonnezrN necrevistonnezr

                                    @nebulon said in 6.3.3 a few quirks:

                                    Ok at least for @atridad the issue was that for some reason old dashboard nginx configs were still around and referencing old SSL certificates which were now purged from the system. This made nginx fail to startup, causing the rest of the side-effects.

                                    That had been my problem as well (more or less): https://forum.cloudron.io/topic/5263/cloudron-does-not-start-after-update-to-6-3-3

                                    ? Offline
                                    ? Offline
                                    A Former User
                                    wrote on last edited by
                                    #24

                                    @necrevistonnezr I just noticed your thread... whoops!

                                    1 Reply Last reply
                                    0
                                    • d19dotcaD Offline
                                      d19dotcaD Offline
                                      d19dotca
                                      wrote on last edited by
                                      #25

                                      @girish - I noticed 6.3.4 is out now... is this one that will again need more time than usual to update, or will it be a typical real quick one with nearly no downtime needed?

                                      --
                                      Dustin Dauncey
                                      www.d19.ca

                                      girishG 2 Replies Last reply
                                      0
                                      • d19dotcaD d19dotca

                                        @girish - I noticed 6.3.4 is out now... is this one that will again need more time than usual to update, or will it be a typical real quick one with nearly no downtime needed?

                                        girishG Do not disturb
                                        girishG Do not disturb
                                        girish
                                        Staff
                                        wrote on last edited by
                                        #26

                                        @d19dotca 6.3.3 to 6.3.4 should be very quick. Only 6.2 -> 6.3 will take a bit to reconfigure containers.

                                        1 Reply Last reply
                                        0
                                        • d19dotcaD d19dotca

                                          @girish - I noticed 6.3.4 is out now... is this one that will again need more time than usual to update, or will it be a typical real quick one with nearly no downtime needed?

                                          girishG Do not disturb
                                          girishG Do not disturb
                                          girish
                                          Staff
                                          wrote on last edited by
                                          #27

                                          @d19dotca Also, it's probably not clear from the UI because the UI is not optimized for the early upgraders. You can see https://git.cloudron.io/cloudron/box/-/blob/master/CHANGES for what exactly is the fix in the patch release.

                                          1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                            • Login

                                            • Don't have an account? Register

                                            • Login or register to search.
                                            • First post
                                              Last post
                                            0
                                            • Categories
                                            • Recent
                                            • Tags
                                            • Popular
                                            • Bookmarks
                                            • Search