Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Ghost
  3. Large JSON file upload getting nginx 413 error

Large JSON file upload getting nginx 413 error

Scheduled Pinned Locked Moved Solved Ghost
nginxuploadcloudflare
17 Posts 4 Posters 3.6k Views 4 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • marcusquinnM marcusquinn

    @tkd I know Cloudflare has a 100MB upload limit if you are using that proxy and other proxies or WAFs may also have limits before the web server. May help, may not, troubleshooting process of elimination there though.

    girishG Do not disturb
    girishG Do not disturb
    girish
    Staff
    wrote on last edited by
    #5

    @marcusquinn Oh, did not know that. In fact, the error message seems exactly the same - https://community.cloudflare.com/t/413-request-entity-too-large-upload-limit-100mb/97630 . So @tkd do you use Cloudflare?

    1 Reply Last reply
    0
    • marcusquinnM Offline
      marcusquinnM Offline
      marcusquinn
      wrote on last edited by
      #6

      @girish Yeah, and it's $20/month per domain for "Pro" to increase that limit.

      Hence, I'm looking at a DNS Made Easy + X4B + BunnyCDN combo as an alternative.

      A little more work to manage separately but the ability to control costs and features more granularly is necessary when managing a variety of domain needs but aiming for a common system among all of them.

      Web Design https://www.evergreen.je
      Development https://brandlight.org
      Life https://marcusquinn.com

      1 Reply Last reply
      0
      • T Offline
        T Offline
        tkd
        wrote on last edited by
        #7

        No I'm not using Cloudflare, Cloudron is hosted at Digital Ocean using and I'm connecting direct to the server, as I mentioned above I'm not aware of any proxy or anything between the browser and Cloudron.

        1 Reply Last reply
        1
        • nebulonN Offline
          nebulonN Offline
          nebulon
          Staff
          wrote on last edited by
          #8

          @tkd do you have a test json file for us to work with? If you have non public data in there, you may also send us a mail to support@cloudron.io

          So far I wasn't finding a quick way to generate an export with over 100M

          T 1 Reply Last reply
          0
          • nebulonN nebulon

            @tkd do you have a test json file for us to work with? If you have non public data in there, you may also send us a mail to support@cloudron.io

            So far I wasn't finding a quick way to generate an export with over 100M

            T Offline
            T Offline
            tkd
            wrote on last edited by
            #9

            @nebulon Sure, it's non public so I will email to support address now thanks.

            1 Reply Last reply
            0
            • nebulonN Offline
              nebulonN Offline
              nebulon
              Staff
              wrote on last edited by
              #10

              Ok, thanks that helped. There are two memory related issues I can reproduce.

              1. Memory limit from Cloudron side: With that json import I had to at least bump up the memory limit for that app instance via the Cloudron dashboard to 1Gb temporarily
              2. It will still fail due to JavaScript heap out of memory in nodejs

              Both issues will result in status code 413, which results in Ghost showing the error message Request is larger than the maximum file size the server allows which is not related to the actual upload size or so, but actually a processing memory requirement issue.

              I don't yet have a solution but just wanted to give an update here.

              1 Reply Last reply
              0
              • T Offline
                T Offline
                tkd
                wrote on last edited by
                #11

                Thanks. This is quite important to us to get this loaded into a Ghost instance. If anyone has any ideas or a temporary work around I'd be happy to know - or if Ghost has another method of importing content that would work with the Cloudron package.

                1 Reply Last reply
                0
                • nebulonN Offline
                  nebulonN Offline
                  nebulon
                  Staff
                  wrote on last edited by
                  #12

                  So the fix, with which I was successfully able to import this file is, to give the javascript engine, which is used by nodejs, more heap space:

                  node --max-old-space-size=4096  current/index.js
                  

                  In this case allowing up to 4Gb. The issue is, I am not sure how to put this properly in the package, as this heavily depends on the set memory limit of the whole app instance.

                  Hopefully we can come up with a good solution later today.

                  T 1 Reply Last reply
                  1
                  • nebulonN nebulon

                    So the fix, with which I was successfully able to import this file is, to give the javascript engine, which is used by nodejs, more heap space:

                    node --max-old-space-size=4096  current/index.js
                    

                    In this case allowing up to 4Gb. The issue is, I am not sure how to put this properly in the package, as this heavily depends on the set memory limit of the whole app instance.

                    Hopefully we can come up with a good solution later today.

                    T Offline
                    T Offline
                    tkd
                    wrote on last edited by
                    #13

                    @nebulon Appreciated. If there is any temporary command I can execute in the app's terminal for now that would make this setting would be fine until you have a full fix.

                    1 Reply Last reply
                    0
                    • nebulonN Offline
                      nebulonN Offline
                      nebulon
                      Staff
                      wrote on last edited by
                      #14

                      We will make a new release which contains a env file at /app/data/env
                      There the max-old-space can be set/overwriten temporarily for the import. The test json in this case worked with a 2Gb limit for me.

                      The file can be changed using the file manger from the cloudron dashboard or webterminal. Once changed, the app has to be restarted.

                      Of course first update your ghost instance to latest package version 3.126.1 🙂

                      1 Reply Last reply
                      3
                      • girishG Do not disturb
                        girishG Do not disturb
                        girish
                        Staff
                        wrote on last edited by
                        #15

                        @tkd Docs are here for the new package - https://cloudron.io/documentation/apps/ghost/#importing

                        1 Reply Last reply
                        1
                        • T Offline
                          T Offline
                          tkd
                          wrote on last edited by
                          #16

                          Thanks guys, managed to get this working with the new package 👍

                          1 Reply Last reply
                          1
                          • marcusquinnM Offline
                            marcusquinnM Offline
                            marcusquinn
                            wrote on last edited by
                            #17

                            Truly heroic support! 🙌 Happy to know this is covered and tested 🙂

                            Web Design https://www.evergreen.je
                            Development https://brandlight.org
                            Life https://marcusquinn.com

                            1 Reply Last reply
                            0
                            Reply
                            • Reply as topic
                            Log in to reply
                            • Oldest to Newest
                            • Newest to Oldest
                            • Most Votes


                            • Login

                            • Don't have an account? Register

                            • Login or register to search.
                            • First post
                              Last post
                            0
                            • Categories
                            • Recent
                            • Tags
                            • Popular
                            • Bookmarks
                            • Search