Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Fire declared in OVH SBG2 datacentre building

    Off-topic
    ovh
    7
    19
    801
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • girish
      girish Staff last edited by girish

      http://travaux.ovh.net/?do=details&id=49471&

      Per https://twitter.com/olesovhcom/status/1369504527544705025, "We recommend to activate your Disaster Recovery Plan."

      P 1 Reply Last reply Reply Quote 4
      • P
        p44 translator @girish last edited by

        Has any Cloudron user been impacted?

        This event should prompt us to use different providers for backups

        mehdi girish 2 Replies Last reply Reply Quote 0
        • mehdi
          mehdi App Dev @p44 last edited by

          @p44 said in Fire declared in OVH SBG2 datacentre building:

          Has any Cloudron user been impacted?

          My company has been impacted, but thankfully not our production servers. : only our CI, and our secondary Object Storage. They are not cloudron servers though.

          P 1 Reply Last reply Reply Quote 0
          • P
            p44 translator @mehdi last edited by

            @mehdi Ok so all your service/servers are operational? Did you performed any recovery/restore action?

            mehdi 1 Reply Last reply Reply Quote 0
            • mehdi
              mehdi App Dev @p44 last edited by

              @p44 All our client-facing services, yes.

              But for internal use, our Gitlab CI is currently down. We are not sure yet if we are part of the destroyed data-center, or just the offline ones, because the OVH manager page is currently down ^^.

              As it's only internal, we are not in a big hurry to recover from backups, but we'll probably do it when the manager is back up.

              P 1 Reply Last reply Reply Quote 2
              • P
                p44 translator @mehdi last edited by

                @mehdi Ok Mehdi thank's a lot

                1 Reply Last reply Reply Quote 1
                • girish
                  girish Staff last edited by

                  Picture of the fire https://pbs.twimg.com/media/EwGqV17XMAMF_wa?format=jpg&name=large

                  https://mobile.twitter.com/abonin_DNA/status/1369538028243456000

                  P P 2 Replies Last reply Reply Quote 1
                  • girish
                    girish Staff @p44 last edited by

                    @p44 yes, some of our users lost their servers. We are helping them restore. Atleast, I only know of those who contacted us on support

                    P 1 Reply Last reply Reply Quote 2
                    • P
                      p44 translator @girish last edited by

                      @girish said in Fire declared in OVH SBG2 datacentre building:

                      Picture of the fire https://pbs.twimg.com/media/EwGqV17XMAMF_wa?format=jpg&name=large

                      https://mobile.twitter.com/abonin_DNA/status/1369538028243456000

                      Very horrible to see.

                      1 Reply Last reply Reply Quote 0
                      • P
                        p44 translator @girish last edited by

                        @girish said in Fire declared in OVH SBG2 datacentre building:

                        @p44 yes, some of our users lost their servers. We are helping them restore. Atleast, I only know of those who contacted us on support

                        I hope everything could be back ASAP operational. Next week we will consolidate our disaster recovery plan.

                        1 Reply Last reply Reply Quote 0
                        • ?
                          A Former User last edited by

                          we are also affected.
                          Cloudron is runnning internal systems like crm and conflunce.

                          I have an offsite backup at exoscale. It is tested some months ago.

                          But know i tried to look at the bucket to get the restore ID (filename) en encounterd an internal Server error at exoscale.

                          The support said that our backup is about 2TB big and because of its structure, which seems to have several delimiters i cant get a listing of backup.

                          I am trying "s3cmd ls s3://***** --recursive" to get prefix and a backup ID

                          so if someone had an similar experience i like to hear tips.

                          Regards Jens

                          P girish 2 Replies Last reply Reply Quote 1
                          • P
                            p44 translator @Guest last edited by

                            @jensbee4 I hope you can restore all of your data and maybe you can find help here to retrieve backup listing

                            1 Reply Last reply Reply Quote 0
                            • girish
                              girish Staff @Guest last edited by

                              @jensbee4 I wrote a quick guide now on how to determine the backup id. Please ignore typos/grammar since this was a bit of a rush job with many people asking how to do this - https://docs.cloudron.io/guides/backup-id/ . For your case, I think you want https://docs.cloudron.io/guides/backup-id/#s3-aws-cli (no need to pass --recursive etc). Let me know if I can improve the guide further.

                              samir ? 2 Replies Last reply Reply Quote 3
                              • samir
                                samir @girish last edited by

                                For people using Cyberduck with OVH, it does not work well because of Strasbourg timed-out connection...

                                I've found an undocumented workaround :

                                • Download OVH Public Cloud Storage Connection Profile
                                • modify the XML file by adding :
                                <key>Region</key>
                                <string>WAW</string>
                                

                                before </dict>

                                Instead of WAW, put your own region target.

                                Cyberduck is up again !

                                Bon courage...

                                1 Reply Last reply Reply Quote 2
                                • P
                                  privsec @girish last edited by

                                  @girish said in Fire declared in OVH SBG2 datacentre building:

                                  Picture of the fire https://pbs.twimg.com/media/EwGqV17XMAMF_wa?format=jpg&name=large

                                  https://mobile.twitter.com/abonin_DNA/status/1369538028243456000

                                  Oh my goodness!! Thats insane!

                                  Im betting OVH is gonna have some good deals coming up

                                  1 Reply Last reply Reply Quote 0
                                  • robi
                                    robi last edited by

                                    Was that a building made of containers for high density compute?

                                    Life of Advanced Technology

                                    1 Reply Last reply Reply Quote 0
                                    • ?
                                      A Former User @girish last edited by

                                      @girish Thanks for the guide. This is a first step.
                                      The problem on my side is i get an internal server error from exoscal. i checked with serveral clients. Will wait if the support can give me a list of buckits....
                                      An important question. Is my infomation correct that every backup is a full backup?
                                      Can ask the exoscal support to delete old objects? Maybe everythink from 2020?

                                      Regards Jens

                                      ? girish 2 Replies Last reply Reply Quote 0
                                      • ?
                                        A Former User @Guest last edited by

                                        @jensbee4
                                        I got the hint to use the exoscale cli. And it seems to work, i am retrieving the backup list.

                                        1 Reply Last reply Reply Quote 0
                                        • girish
                                          girish Staff @Guest last edited by

                                          @jensbee4 said in Fire declared in OVH SBG2 datacentre building:

                                          The problem on my side is i get an internal server error from exoscal. i checked with serveral clients. Will wait if the support can give me a list of buckits....

                                          Oh, that's unfortunate 😞 I am surprised how an object storage service can fail with just 2TB.

                                          An important question. Is my infomation correct that every backup is a full backup?

                                          It's not! Some backups are the backup of a single app. This is during right before we update an app. I created a task to make this clearer (https://git.cloudron.io/cloudron/box/-/issues/775).

                                          If you want to give a person instructions on how to cleanup backups, you can tell them that if a directory contains a file named box_xxx, then it's a full backup. Otherwise, it's not. In general, if you had backups for this year, you can safely remove everything from 2020 I guess. Alternately, the backend people must have some mechanism to stash those things in some other bucket temporarily instead of deleting it.

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Powered by NodeBB