Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Feature Requests
  3. Improve DNS updates to avoid rate limits

Improve DNS updates to avoid rate limits

Scheduled Pinned Locked Moved Solved Feature Requests
dns
18 Posts 4 Posters 2.9k Views 4 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • L Offline
    L Offline
    LoudLemur
    wrote on last edited by
    #9

    Route 53 (from Amazon) and Bunny.net are mentioned in the Cloudron blog on running a Cloudron at home.
    https://blog.cloudron.io/installing-cloudron-on-a-home-server/

    If you visit eg Bunny, they don't seem to sell domain names, in the way that other companies do, eg. Epik. How would you get a domain name from Bunny or route 53?

    girishG 1 Reply Last reply
    0
    • L LoudLemur

      Route 53 (from Amazon) and Bunny.net are mentioned in the Cloudron blog on running a Cloudron at home.
      https://blog.cloudron.io/installing-cloudron-on-a-home-server/

      If you visit eg Bunny, they don't seem to sell domain names, in the way that other companies do, eg. Epik. How would you get a domain name from Bunny or route 53?

      girishG Offline
      girishG Offline
      girish
      Staff
      wrote on last edited by
      #10

      @LoudLemur I guess you are posting on the wrong thread?

      L 1 Reply Last reply
      0
      • girishG girish

        @LoudLemur I guess you are posting on the wrong thread?

        L Offline
        L Offline
        LoudLemur
        wrote on last edited by
        #11

        @girish said in Improve DNS updates to avoid rate limits:

        @LoudLemur I guess you are posting on the wrong thread?

        I thought I would throw the question about Bunny and Route 53 in here, since people seem to be familiar with the services and they are up for discussion. Maybe it would be better to create another thread. ...

        1 Reply Last reply
        0
        • girishG Offline
          girishG Offline
          girish
          Staff
          wrote on last edited by
          #12

          With a bit of hunting, I found https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/DNSLimitations.html#limits-api-requests . It says "Five requests per second per AWS account per Region" . Looks like they recommend implementing a backoff as the solution. I will try to add this in our route53 backend.

          1 Reply Last reply
          2
          • girishG Offline
            girishG Offline
            girish
            Staff
            wrote on last edited by
            #13

            Retry added with https://git.cloudron.io/cloudron/box/-/commit/37ed87f9c18c8d4f37a86e5503e6cc2870b215e4

            1 Reply Last reply
            2
            • girishG girish marked this topic as a question on
            • girishG girish has marked this topic as solved on
            • robiR Offline
              robiR Offline
              robi
              wrote on last edited by
              #14

              Okay, thanks for the retry delay patch, can you apply it to the demo server?

              This, however, does not solve the topic though, for which the list compare approach makes a large list with one change just do one change as an atomic subset, not the entire list as a global re-set.

              In Multi Cloudron this could potentially create API storms.
              We're getting close to this with the large netblock lists etc.

              Please think ahead on this one.

              Conscious tech

              girishG 1 Reply Last reply
              0
              • robiR robi

                Okay, thanks for the retry delay patch, can you apply it to the demo server?

                This, however, does not solve the topic though, for which the list compare approach makes a large list with one change just do one change as an atomic subset, not the entire list as a global re-set.

                In Multi Cloudron this could potentially create API storms.
                We're getting close to this with the large netblock lists etc.

                Please think ahead on this one.

                girishG Offline
                girishG Offline
                girish
                Staff
                wrote on last edited by girish
                #15

                @robi I have patched the demo server.

                As for the optimization: Cloudron checks if the record is up to date first and only updates it if it is not up to date. It's not a blind update each time. This is the best we can do without complex state tracking.

                To write code such as "continue from where it last errored" and "update whatever you can right now" and "only updated what is minimally required" etc , requires Cloudron to track state . In practically all cases, this state tracking is not needed. The most common case (by which is mean 99.9999%) is that there are say < 5 domains for an app. So, all this code and state tracking to reduce some API requests? It's not worth the complexity and it's not like we are generating some 1000 requests. Also, think about all the UI buttons and error messages that now need to be created just for this. You need a button to say "continue" , messages that show what exactly went wrong and what domains did not update and why exactly and so on.

                robiR 1 Reply Last reply
                0
                • girishG girish

                  @robi I have patched the demo server.

                  As for the optimization: Cloudron checks if the record is up to date first and only updates it if it is not up to date. It's not a blind update each time. This is the best we can do without complex state tracking.

                  To write code such as "continue from where it last errored" and "update whatever you can right now" and "only updated what is minimally required" etc , requires Cloudron to track state . In practically all cases, this state tracking is not needed. The most common case (by which is mean 99.9999%) is that there are say < 5 domains for an app. So, all this code and state tracking to reduce some API requests? It's not worth the complexity and it's not like we are generating some 1000 requests. Also, think about all the UI buttons and error messages that now need to be created just for this. You need a button to say "continue" , messages that show what exactly went wrong and what domains did not update and why exactly and so on.

                  robiR Offline
                  robiR Offline
                  robi
                  wrote on last edited by
                  #16

                  @girish Thanks! Will test ;-]

                  Q: Why not trust the last state?

                  You already have a button to re-sync DNS in case that becomes an issue.

                  Q: Why do you need any new UI buttons or error messages when it's "handled" in the background transparently?

                  Isn't it iterative retry to eventual completion even if failures occur?

                  Conscious tech

                  girishG 1 Reply Last reply
                  0
                  • robiR robi

                    @girish Thanks! Will test ;-]

                    Q: Why not trust the last state?

                    You already have a button to re-sync DNS in case that becomes an issue.

                    Q: Why do you need any new UI buttons or error messages when it's "handled" in the background transparently?

                    Isn't it iterative retry to eventual completion even if failures occur?

                    girishG Offline
                    girishG Offline
                    girish
                    Staff
                    wrote on last edited by
                    #17

                    @robi said in Improve DNS updates to avoid rate limits:

                    Q: Why not trust the last state?

                    You already have a button to re-sync DNS in case that becomes an issue.

                    Maybe I am not completely understanding. Let's say you have 5 domains right now and say 2 failed . The app now says 'Running', but I also assume it has show some message or error somewhere that 2 failed. Otherwise, people will say it didn't really succeed. To track that 2 failed, we have to maintain state. But ok, let's say we ignore this. User adds a domain. Now, we only try this new domain. The 2 failed are silently ignored. And maybe user adds more and more failures add up all silently ignored. There's no way to know what has succeeded and what has failed unless we maintain state.

                    robiR 1 Reply Last reply
                    0
                    • girishG girish

                      @robi said in Improve DNS updates to avoid rate limits:

                      Q: Why not trust the last state?

                      You already have a button to re-sync DNS in case that becomes an issue.

                      Maybe I am not completely understanding. Let's say you have 5 domains right now and say 2 failed . The app now says 'Running', but I also assume it has show some message or error somewhere that 2 failed. Otherwise, people will say it didn't really succeed. To track that 2 failed, we have to maintain state. But ok, let's say we ignore this. User adds a domain. Now, we only try this new domain. The 2 failed are silently ignored. And maybe user adds more and more failures add up all silently ignored. There's no way to know what has succeeded and what has failed unless we maintain state.

                      robiR Offline
                      robiR Offline
                      robi
                      wrote on last edited by
                      #18

                      @girish You're right, I think not.

                      Currently the app is NOT in a 'Running' state if any failures occur.

                      An alternate method would be to bring up the app anyway (as long as the primary subdomain is up, and continue to resolve the failed API calls until some 'threshold' (then alert) or it succeeds and all is well.

                      No need to ignore the failed ones, they just try again async as with your API rate backoff code, or at the next addition of a new sub if you want to be less proactive and more lazy. (The non working subs, if used will draw attention back to the app anyway.)

                      Conscious tech

                      1 Reply Last reply
                      0
                      Reply
                      • Reply as topic
                      Log in to reply
                      • Oldest to Newest
                      • Newest to Oldest
                      • Most Votes


                      • Login

                      • Don't have an account? Register

                      • Login or register to search.
                      • First post
                        Last post
                      0
                      • Categories
                      • Recent
                      • Tags
                      • Popular
                      • Bookmarks
                      • Search