Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Aws Backup error : EPIPE HTTP Code : NetworkingError

    Support
    bacups aws
    3
    10
    582
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      CarbonBee last edited by girish

      Hi!

      I recently setup my Cloudron server to backup with aws s3 everyday. The last few days, the backup ended when trying to upload my nextcloud backup (which weights more than 100GB).
      The logs report the following (for security purpose, I removed the domain name, the application ID and the aws id) :

      Oct 20 03:29:26 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 392M@1MBps (cloud.mydomain.fr)"}
      Oct 20 03:29:37 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 104M@0MBps (cloud.mydomain.fr)"}
      Oct 20 03:30:41 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 392M@1MBps (cloud.mydomain.fr)"}
      Oct 20 03:30:55 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 1404M@2MBps (cloud.mydomain.fr)"}
      Oct 20 03:34:06 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 104M@0MBps (cloud.mydomain.fr)"}
      Oct 20 03:34:15 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 877M@1MBps (cloud.mydomain.fr)"}
      Oct 20 03:34:37 box:tasks 5308: {"percent":87.66666666666667,"message":"Uploading backup 104M@0MBps (cloud.mydomain.fr)"}
      Oct 20 03:37:31 box:shell backup-snapshot/app_<appId> (stdout): 2020-10-20T01:37:18.907Z box:storage/s3 Error uploading [snapshot/app_<appId>.tar.gz.enc]: s3 upload error. { Error: write EPIPE
      at WriteWrap.afterWrite [as oncomplete] (net.js:789:14)
      message: 'write EPIPE',
      errno: 'EPIPE',
      code: 'NetworkingError',
      syscall: 'write',
      region: 'eu-west-3',
      hostname: '<s3Id>.s3.eu-west-3.amazonaws.com',
      retryable: true,
      time: 2020-10-20T01:36:48.195Z }
      Oct 20 03:37:19 box:backupupload upload completed. error: { BoxError: Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError
      at ManagedUpload.callback (/home/yellowtent/box/src/storage/s3.js:130:33)
      at ManagedUpload.cleanup (/home/yellowtent/box/node_modules/aws-sdk/lib/s3/managed_upload.js:629:10)
      at Response.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/s3/managed_upload.js:566:28)
      at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:364:18)
      at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
      at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:683:14)
      at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
      name: 'BoxError',
      reason: 'External Error',
      details: {},
      message:
      'Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError' }
      Oct 20 03:37:41 box:backups runBackupUpload: result - {"result":"Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError"}
      Oct 20 03:39:26 box:shell backup-snapshot/app_<appId> code: 50, signal: null
      Oct 20 03:39:30 box:backups cloud.mydomain.fr Unable to backup { BoxError: Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError
      at /home/yellowtent/box/src/backups.js:863:29
      at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
      at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)
      at ChildProcess.emit (events.js:198:13)
      at ChildProcess.EventEmitter.emit (domain.js:448:20)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
      name: 'BoxError',
      reason: 'External Error',
      details: {},
      message:
      'Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError' }
      Oct 20 03:39:31 box:taskworker Task took 16766.54 seconds
      Oct 20 03:39:31 box:tasks setCompleted - 5308: {"result":null,"error":{"stack":"BoxError: Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError\n at /home/yellowtent/box/src/backups.js:863:29\n at f (/home/yellowtent/box/node_modules/once/once.js:25:25)\n at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)\n at ChildProcess.emit (events.js:198:13)\n at ChildProcess.EventEmitter.emit (domain.js:448:20)\n at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)","name":"BoxError","reason":"External Error","details":{},"message":"Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError"}}
      Oct 20 03:39:32 box:tasks 5308: {"percent":100,"result":null,"error":{"stack":"BoxError: Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError\n at /home/yellowtent/box/src/backups.js:863:29\n at f (/home/yellowtent/box/node_modules/once/once.js:25:25)\n at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)\n at ChildProcess.emit (events.js:198:13)\n at ChildProcess.EventEmitter.emit (domain.js:448:20)\n at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)","name":"BoxError","reason":"External Error","details":{},"message":"Error uploading snapshot/app_<appId>.tar.gz.enc. Message: write EPIPE HTTP Code: NetworkingError"}}
      

      I tried with 10MB and 100MB upload part size, but this does not seems to change anything.
      Do you have any idea on what's going on?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • girish
        girish Staff last edited by

        The issue was that we had a 5min upload timeout for each chunk in code. Removing the timeout made it work. Fix is in the next release.

        jordanurbs 1 Reply Last reply Reply Quote 0
        • girish
          girish Staff last edited by

          @CarbonBee Are you able to reproduce it all the time/frequently? It seems maybe the network on the server fluctuated.

          1 Reply Last reply Reply Quote 0
          • C
            CarbonBee last edited by

            It's always doing that. I haven't been able to make a full backup for days. This begins to be critical.

            1 Reply Last reply Reply Quote 0
            • girish
              girish Staff last edited by

              @CarbonBee Atleast the error seems to indicate that it's a networking error. Could it be that your VPS provider is throttling uploads of 100GB ?

              C 1 Reply Last reply Reply Quote 0
              • C
                CarbonBee @girish last edited by

                @girish I tried to backup only the nextcloud, which generated the same error.
                I have no idea on any upload limitation. My server is hosted by Kimsufi, an OVH discounter.

                1 Reply Last reply Reply Quote 0
                • C
                  CarbonBee last edited by

                  Well, a technician from Kimsufi just told me that there are no upload limit on my server. I don't really understand what's going on, because on more or less the same setup on a different server, it works fine.
                  Is there a way to track it down and find out where the issue come from?

                  1 Reply Last reply Reply Quote 0
                  • girish
                    girish Staff last edited by

                    @CarbonBee I can't make out from the error what the error is. Are you able to contact us at support@cloudron.io and maybe give us ssh access?

                    1 Reply Last reply Reply Quote 0
                    • girish
                      girish Staff last edited by

                      The issue was that we had a 5min upload timeout for each chunk in code. Removing the timeout made it work. Fix is in the next release.

                      jordanurbs 1 Reply Last reply Reply Quote 0
                      • jordanurbs
                        jordanurbs @girish last edited by

                        @girish is this fix and next release out yet? I have been consistently getting this error as of the last 2-3 weeks or so, out of the blue.

                        girish 1 Reply Last reply Reply Quote 0
                        • girish
                          girish Staff @jordanurbs last edited by

                          @jordanurbs Yes, this is in 6.0.1. Are you in 6.0.1 ? https://git.cloudron.io/cloudron/box/-/commit/bedcd6fccf58830b316318699375bc1f582a5d7a has the fix. Do you have any error logs?

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Powered by NodeBB