Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. Backups failing for 3 weeks

Backups failing for 3 weeks

Scheduled Pinned Locked Moved Support
backups
9 Posts 3 Posters 1.6k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A Offline
    A Offline
    adrw
    wrote on last edited by girish
    #1

    My server is on Cloudron 6.2.4 and backups have been failing for the past 3 weeks with error logs that don't seem to indicate any of the past bugs I've encountered (and have since been fixed – many thanks to @girish and others). I've tried reducing memory usage of services like Email (turning off the full text Solr search) thinking Cloudron was running out of memory but nothing seems to help the backup proceed.

    Any ideas?

    Mar 13 14:36:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:36:16.877Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25609544 external: 33118796
    Mar 13 14:36:16 box:backupupload v8 heap : used 25609544 total: 27140096 max: 4076863488
    Mar 13 14:36:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:36:46.877Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25641704 external: 33118796
    Mar 13 14:36:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:36:46.877Z box:backupupload v8 heap : used 25641704 total: 27140096 max: 4076863488
    Mar 13 14:37:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:37:16.878Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25674104 external: 33118796
    Mar 13 14:37:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:37:16.878Z box:backupupload v8 heap : used 25674104 total: 27140096 max: 4076863488
    Mar 13 14:37:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:37:46.878Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25706264 external: 33118796
    Mar 13 14:37:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:37:46.878Z box:backupupload v8 heap : used 25706264 total: 27140096 max: 4076863488
    Mar 13 14:38:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:38:16.880Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25738424 external: 33118796
    Mar 13 14:38:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:38:16.880Z box:backupupload v8 heap : used 25738424 total: 27140096 max: 4076863488
    Mar 13 14:38:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:38:46.880Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25770960 external: 33118796
    Mar 13 14:38:46 box:backupupload v8 heap : used 25770960 total: 27140096 max: 4076863488
    Mar 13 14:39:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:39:16.882Z box:backupupload process: rss: 114876416 heapTotal: 27140096 heapUsed: 25803120 external: 33118796
    Mar 13 14:39:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:39:16.882Z box:backupupload v8 heap : used 25803120 total: 27140096 max: 4076863488
    Mar 13 14:39:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:39:46.881Z box:backupupload process: rss: 115146752 heapTotal: 27140096 heapUsed: 25835520 external: 33118796
    Mar 13 14:39:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:39:46.881Z box:backupupload v8 heap : used 25835520 total: 27140096 max: 4076863488
    Mar 13 14:40:16 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:40:16.883Z box:backupupload process: rss: 115146752 heapTotal: 27140096 heapUsed: 25867680 external: 33118796
    Mar 13 14:40:16 box:backupupload v8 heap : used 25867680 total: 27140096 max: 4076863488
    Mar 13 14:40:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:40:46.883Z box:backupupload process: rss: 115146752 heapTotal: 27140096 heapUsed: 25900048 external: 33118796
    Mar 13 14:40:46 box:shell backup-snapshot/app_62c2ebf9-92ea-4038-84db-097e7fe100a7 (stdout): 2021-03-13T19:40:46.883Z box:backupupload v8 heap : used 25900048 total: 27140096 max: 4076863488
    
    1 Reply Last reply
    0
    • nebulonN Offline
      nebulonN Offline
      nebulon
      Staff
      wrote on last edited by
      #2

      What is the failure you are seeing, like does it just time-out or is there a crash?

      The logs you pasted don't seem to contain actual errors to investigate further.

      A 1 Reply Last reply
      0
      • nebulonN nebulon

        What is the failure you are seeing, like does it just time-out or is there a crash?

        The logs you pasted don't seem to contain actual errors to investigate further.

        A Offline
        A Offline
        adrw
        wrote on last edited by
        #3

        @nebulon There isn't an immediately visible stack trace, it seems to time out. I've uploaded the full logs here

        nebulonN 1 Reply Last reply
        0
        • A adrw

          @nebulon There isn't an immediately visible stack trace, it seems to time out. I've uploaded the full logs here

          nebulonN Offline
          nebulonN Offline
          nebulon
          Staff
          wrote on last edited by
          #4

          @adrw hm I see, so the snapshotting is working and then during backup rotation, uploading the tarball it just stops without further information looks like:

          2021-03-16T20:59:50.991Z box:tasks 3493: {"percent":64.63636363636364,"message":"Copying part 231 - /my-adrw-xyz/snapshot/app_c4dd3c6d-806e-454b-beba-4cdd3f29865a.tar.gz.enc bytes=246960619520-248034361343 (cloud.alxdr.ca)"}
          

          Which storage provider is this? Maybe you can increase the part-size for s3 in the backup configs (if that applies).

          A 2 Replies Last reply
          0
          • nebulonN nebulon

            @adrw hm I see, so the snapshotting is working and then during backup rotation, uploading the tarball it just stops without further information looks like:

            2021-03-16T20:59:50.991Z box:tasks 3493: {"percent":64.63636363636364,"message":"Copying part 231 - /my-adrw-xyz/snapshot/app_c4dd3c6d-806e-454b-beba-4cdd3f29865a.tar.gz.enc bytes=246960619520-248034361343 (cloud.alxdr.ca)"}
            

            Which storage provider is this? Maybe you can increase the part-size for s3 in the backup configs (if that applies).

            A Offline
            A Offline
            adrw
            wrote on last edited by
            #5

            @nebulon I'm using BackblazeB2, thanks for looking into this!

            1 Reply Last reply
            0
            • nebulonN nebulon

              @adrw hm I see, so the snapshotting is working and then during backup rotation, uploading the tarball it just stops without further information looks like:

              2021-03-16T20:59:50.991Z box:tasks 3493: {"percent":64.63636363636364,"message":"Copying part 231 - /my-adrw-xyz/snapshot/app_c4dd3c6d-806e-454b-beba-4cdd3f29865a.tar.gz.enc bytes=246960619520-248034361343 (cloud.alxdr.ca)"}
              

              Which storage provider is this? Maybe you can increase the part-size for s3 in the backup configs (if that applies).

              A Offline
              A Offline
              adrw
              wrote on last edited by
              #6

              @nebulon I tried increasing backup chunk size from 256mb to 1GB and backups are still failing similarly, any idea or other configuration I should be tweaking?

              nebulonN 1 Reply Last reply
              0
              • A adrw

                @nebulon I tried increasing backup chunk size from 256mb to 1GB and backups are still failing similarly, any idea or other configuration I should be tweaking?

                nebulonN Offline
                nebulonN Offline
                nebulon
                Staff
                wrote on last edited by
                #7

                @adrw not really, maybe we have to debug this directly to get more information.
                If you want you can enable remote support for us and send us a mail with your dashboard domain and reference to this thread at support@cloudron.io

                A 1 Reply Last reply
                1
                • scookeS Offline
                  scookeS Offline
                  scooke
                  wrote on last edited by
                  #8

                  FWIW, I've twice had backup failures when using Cloudron (at previous versions), and both times it was due to some image in the backup. One instance was definitely a Wordpress install, and the other some kind of crm, I think on the LAMP app. I never did figure out what it was exactly, but once I deleted one specific jpg, my backups resumed with no problem. The backup problems was an unexplained and unclear stall/hang, and the other was a failure. But neither, I am quite certain, had any definite pointer to the image as the problem.
                  As I think about it a bit more, the Wordpress image was actually an image hosted remotely on my cloudinary account. I reckon the backup was trying to pull that image and was stalling.
                  You never know.

                  A life lived in fear is a life half-lived

                  1 Reply Last reply
                  0
                  • nebulonN nebulon

                    @adrw not really, maybe we have to debug this directly to get more information.
                    If you want you can enable remote support for us and send us a mail with your dashboard domain and reference to this thread at support@cloudron.io

                    A Offline
                    A Offline
                    adrw
                    wrote on last edited by
                    #9

                    @nebulon I enabled remote debugging and emailed, look forward to hearing back

                    1 Reply Last reply
                    0
                    Reply
                    • Reply as topic
                    Log in to reply
                    • Oldest to Newest
                    • Newest to Oldest
                    • Most Votes


                    • Login

                    • Don't have an account? Register

                    • Login or register to search.
                    • First post
                      Last post
                    0
                    • Categories
                    • Recent
                    • Tags
                    • Popular
                    • Bookmarks
                    • Search