Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Minio Client

    Minio
    3
    7
    522
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      RoboMod last edited by

      Are there any plans to use the minio client for minio backups? I'm asking because I have trouble uploading a "larger" dataset of about 10 GB. The problem is that the backup always runs to a certain file because it uploads every single file again even if there is no change. The minio client provides a "mirror" function which would deal better with that using differential uploads.

      1 Reply Last reply Reply Quote 0
      • nebulon
        nebulon Staff last edited by

        Never heard of the minio client mirror function, but the cloudron backup rsyncer would also only upload the changed files, however only once an initial full backup is finished completely, which probably is then not the case for you. Are there any errors reported for the backup?

        R 1 Reply Last reply Reply Quote 0
        • R
          RoboMod @nebulon last edited by

          @nebulon On Cloudron's side I get the error: 'Connection timed out after 300000ms HTTP Code: TimeoutError'. Looking on my minio server I get: 'msg="Unable to create object part." cause="#has incomplete body"' and 'msg="Failed to create object <filename>" cause="unexpected EOF"'.

          1 Reply Last reply Reply Quote 0
          • nebulon
            nebulon Staff last edited by

            That looks like a bug itself and should be fixed instead of moving to some other syncing approach. Is there anything suspicious about this file (size, filename, ...)? Also does this happen always to the same file?

            R 1 Reply Last reply Reply Quote 0
            • R
              RoboMod @nebulon last edited by

              @nebulon It seems to happen on different files but till now always in the same directory. Right now, I'm running an app backup only of this nextcloud instance. I'll report back when this is finished.

              1 Reply Last reply Reply Quote 0
              • R
                RoboMod last edited by

                Okay. So, now I let cloudron try to updated and thereby backup this nextcloud installation again and again. I always get the following error for many files:

                2019-03-03T18:35:47.350Z box:shell backup-snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2 (stdout): 2019-03-03T18:35:47.348Z box:storage/s3 Error uploading [snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>]: s3 upload error. { TimeoutError: Connection timed out after 300000ms
                    at ClientRequest.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/http/node.js:83:34)
                    at Object.onceWrapper (events.js:313:30)
                    at emitNone (events.js:106:13)
                    at ClientRequest.emit (events.js:208:7)
                    at Socket.emitTimeout (_http_client.js:706:34)
                    at Object.onceWrapper (events.js:313:30)
                    at emitNone (events.js:106:13)
                    at Socket.emit (events.js:208:7)
                    at Socket._onTimeout (net.js:410:8)
                    at ontimeout (timers.js:498:11)
                  message: 'Connection timed out after 300000ms',
                  code: 'TimeoutError',
                  time: 2019-03-03T18:35:47.342Z,
                  region: 'us-east-1',
                  hostname: 'localhost',
                  retryable: true }
                

                Then I get these errors for only a file:

                2019-03-03T18:50:06.563Z box:shell backup-snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2 (stdout): 2019-03-03T18:50:06.286Z box:syncer Done processing adds { BackupsError: Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError
                    at ManagedUpload.callback (/home/yellowtent/box/src/storage/s3.js:151:33)
                    at Response.finishSinglePart (/home/yellowtent/box/node_modules/aws-sdk/lib/s3/managed_upload.js:661:28)
                    at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:364:18)
                    at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
                    at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
                    at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:683:14)
                    at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
                    at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
                    at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
                    at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)
                  name: 'BackupsError',
                  reason: 'external error',
                  message: 'Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError' }
                
                2019-03-03T18:50:06.568Z box:shell backup-snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2 (stdout): 2019-03-03T18:50:06.564Z box:backupupload upload completed with error { BackupsError: Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError
                    at /home/yellowtent/box/src/backups.js:373:36
                    at /home/yellowtent/box/src/syncer.js:166:31
                    at /home/yellowtent/box/node_modules/async/dist/async.js:473:16
                    at iterateeCallback (/home/yellowtent/box/node_modules/async/dist/async.js:988:17)
                    at /home/yellowtent/box/node_modules/async/dist/async.js:969:16
                    at /home/yellowtent/box/node_modules/async/dist/async.js:4617:26
                    at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
                    at /home/yellowtent/box/src/backups.js:368:21
                    at ManagedUpload.callback (/home/yellowtent/box/src/storage/s3.js:151:24)
                    at Response.finishSinglePart (/home/yellowtent/box/node_modules/aws-sdk/lib/s3/managed_upload.js:661:28)
                  name: 'BackupsError',
                  reason: 'external error',
                  message: 'Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError' }
                

                And in the end the backup crashed with:

                2019-03-03T18:50:06.569Z box:shell backup-snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2 (stdout): 2019-03-03T18:50:06.569Z box:backupupload upload completed
                
                2019-03-03T18:50:06.706Z box:backups runBackupUpload: result - [object Object]
                2019-03-03T18:50:06.783Z box:shell backup-snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2 code: 50, signal: null
                2019-03-03T18:50:06.895Z box:apptask nextcloud.xeac.de update aborted because backup failed { BackupsError: Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError
                    at /home/yellowtent/box/src/backups.js:682:29
                    at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
                    at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:66:9)
                    at emitTwo (events.js:126:13)
                    at ChildProcess.emit (events.js:214:7)
                    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
                  name: 'BackupsError',
                  reason: 'external error',
                  message: 'Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError',
                  backupError: true }
                2019-03-03T18:50:06.896Z box:apptask nextcloud.xeac.de updating app with values: {"installationState":"installed","installationProgress":"","health":null,"updateConfig":null}
                2019-03-03T18:50:07.095Z box:apptask Apptask completed with error { BackupsError: Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError
                    at /home/yellowtent/box/src/backups.js:682:29
                    at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
                    at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:66:9)
                    at emitTwo (events.js:126:13)
                    at ChildProcess.emit (events.js:214:7)
                    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
                  name: 'BackupsError',
                  reason: 'external error',
                  message: 'Error uploading snapshot/app_836a8eb8-4f45-4099-aac5-83d6c25884c2/data/<path>. Message: Connection timed out after 300000ms HTTP Code: TimeoutError',
                  backupError: true }
                2019-03-03T18:50:07.096Z box:apptask Apptask completed for 836a8eb8-4f45-4099-aac5-83d6c25884c2
                
                1 Reply Last reply Reply Quote 0
                • girish
                  girish Staff last edited by girish

                  @RoboMod Sure, sorry for the no response. From the logs in the other post, the error is Message: Connection timed out after 300000ms HTTP Code: TimeoutError. Maybe the server becomes unreachable because of some network error? Is that possible in your setup?

                  1 Reply Last reply Reply Quote 0
                  • First post
                    Last post
                  Powered by NodeBB