Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

Cloudron Forum

Apps | Demo | Docs | Install

DNS - DigitalOcean - ERROR 429 - Too Many Requests

Scheduled Pinned Locked Moved Unsolved Support
9 Posts 3 Posters 120 Views
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • BrutalBirdieB Offline
    BrutalBirdieB Offline
    BrutalBirdie Staff
    wrote on last edited by
    #1

    @staff ping

    2023-07-24T10:18:01.235Z box:settings initCache: pre-load settings
    2023-07-24T10:18:01.249Z box:taskworker Starting task 738. Logs are at /home/yellowtent/platformdata/logs/tasks/738.log
    2023-07-24T10:18:01.258Z box:tasks update 738: {"percent":1,"message":"Updating DNS of DOMAIN.TLD"}
    2023-07-24T10:18:01.259Z box:dns registerLocations: Will register [{"subdomain":"my","domain":"DOMAIN.TLD"},{"subdomain":"mail","domain":"DOMAIN.TLD"},{"subdomain":"sogo","domain":"DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"mail","certificate":null,"fqdn":"mail.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"webmail","certificate":null,"fqdn":"webmail.DOMAIN.TLD"},{"subdomain":"short","domain":"DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"k","certificate":null,"fqdn":"k.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"kurz","certificate":null,"fqdn":"kurz.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"kutt","certificate":null,"fqdn":"kutt.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"s","certificate":null,"fqdn":"s.DOMAIN.TLD"},{"subdomain":"vault","domain":"DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"pass","certificate":null,"fqdn":"pass.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"passwoerter","certificate":null,"fqdn":"passwoerter.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"password","certificate":null,"fqdn":"password.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"passwords","certificate":null,"fqdn":"passwords.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"passwort","certificate":null,"fqdn":"passwort.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"pw","certificate":null,"fqdn":"pw.DOMAIN.TLD"},{"subdomain":"collabora","domain":"DOMAIN.TLD"},{"subdomain":"roundcube","domain":"DOMAIN.TLD"},{"subdomain":"nextcloud","domain":"DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"cloud","certificate":null,"fqdn":"cloud.DOMAIN.TLD"},{"subdomain":"meet","domain":"DOMAIN.TLD"},{"subdomain":"stats","domain":"DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"matomo","certificate":null,"fqdn":"matomo.DOMAIN.TLD"},{"subdomain":"oc-test1","domain":"DOMAIN.TLD"},{"subdomain":"hedgedoc","domain":"DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"pad","certificate":null,"fqdn":"pad.DOMAIN.TLD"},{"domain":"DOMAIN.TLD","subdomain":"text","certificate":null,"fqdn":"text.DOMAIN.TLD"},{"subdomain":"chat","domain":"DOMAIN.TLD"}] with options {"overwriteDns":true}
    2023-07-24T10:18:01.262Z box:sysinfo/generic getServerIPv4: querying ipv4.api.cloudron.io to get server IPv4
    2023-07-24T10:18:01.665Z box:sysinfo/generic getServerIPv6: querying ipv6.api.cloudron.io to get server IPv6
    2023-07-24T10:18:02.018Z box:tasks update 738: {"message":"Registering location: my.DOMAIN.TLD"}
    2023-07-24T10:18:02.023Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with my and type CNAME
    2023-07-24T10:18:02.846Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with my and type A
    2023-07-24T10:18:03.516Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with my and type AAAA
    2023-07-24T10:18:04.164Z box:tasks update 738: {"message":"Registering location: mail.DOMAIN.TLD"}
    2023-07-24T10:18:04.165Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with mail and type CNAME
    2023-07-24T10:18:04.837Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with mail and type A
    2023-07-24T10:18:05.566Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with mail and type AAAA
    2023-07-24T10:18:06.269Z box:tasks update 738: {"message":"Registering location: sogo.DOMAIN.TLD"}
    2023-07-24T10:18:06.271Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with sogo and type CNAME
    2023-07-24T10:18:06.929Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with sogo and type A
    2023-07-24T10:18:07.574Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with sogo and type AAAA
    2023-07-24T10:18:08.247Z box:tasks update 738: {"message":"Registering location: mail.DOMAIN.TLD"}
    2023-07-24T10:18:08.249Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with mail and type CNAME
    2023-07-24T10:18:08.895Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with mail and type A
    2023-07-24T10:18:09.786Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with mail and type AAAA
    2023-07-24T10:18:10.691Z box:tasks update 738: {"message":"Registering location: webmail.DOMAIN.TLD"}
    2023-07-24T10:18:10.692Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with webmail and type CNAME
    2023-07-24T10:18:11.358Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with webmail and type A
    2023-07-24T10:18:12.268Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with webmail and type AAAA
    2023-07-24T10:18:13.007Z box:tasks update 738: {"message":"Registering location: short.DOMAIN.TLD"}
    2023-07-24T10:18:13.008Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with short and type CNAME
    2023-07-24T10:18:13.851Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with short and type A
    2023-07-24T10:18:14.502Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with short and type AAAA
    2023-07-24T10:18:15.129Z box:tasks update 738: {"message":"Registering location: k.DOMAIN.TLD"}
    2023-07-24T10:18:15.130Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with k and type CNAME
    2023-07-24T10:18:15.812Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with k and type A
    2023-07-24T10:18:16.486Z box:dns/digitalocean getInternal: getting dns records of DOMAIN.TLD with k and type AAAA
    2023-07-24T10:18:17.534Z box:dns registerLocation: Get error. retryable: true. DigitalOcean DNS error 429 {"id":"Too Many Requests","message":"Too many requests"}
    2023-07-24T10:18:17.535Z box:tasks setCompleted - 738: {"result":{"errors":[{"domain":"DOMAIN.TLD","message":"DigitalOcean DNS error 429 {\"id\":\"Too Many Requests\",\"message\":\"Too many requests\"}"}]},"error":null}
    2023-07-24T10:18:17.535Z box:tasks update 738: {"percent":100,"result":{"errors":[{"domain":"DOMAIN.TLD","message":"DigitalOcean DNS error 429 {\"id\":\"Too Many Requests\",\"message\":\"Too many requests\"}"}]},"error":null}
    2023-07-24T10:18:17.535Z box:taskworker Task took 16.346 seconds
    

    I noticed this on multiple Cloudron instances that use DO DNS when syncing DNS records.
    This got noticed after a customer fiddled with the SPF record for some reason and me wanting to re-sync the DNS.

    Like my work? Consider donating a drink. Cheers!

    1 Reply Last reply
    0
  • nebulonN Offline
    nebulonN Offline
    nebulon Staff
    wrote on last edited by
    #2

    how many domains or rather DNS records are we roughly talking about? Maybe DO has changed their api rate-limits recently

    1 Reply Last reply
    0
  • BrutalBirdieB Offline
    BrutalBirdieB Offline
    BrutalBirdie Staff
    wrote on last edited by BrutalBirdie
    #3

    15x apps / subdomains so 30x Domain Records since A and AAAA records.
    To that coming txt records for mail and all.

    Like my work? Consider donating a drink. Cheers!

    1 Reply Last reply
    0
  • nebulonN Offline
    nebulonN Offline
    nebulon Staff
    wrote on last edited by
    #4

    The rate-limit seem a lot higher than this https://docs.digitalocean.com/reference/api/api-reference/#section/Introduction/Rate-Limit

    When did this start and since you mentioned multiple Cloudrons, are those using maybe the same api token?

    1 Reply Last reply
    0
  • nebulonN nebulon marked this topic as a question on
  • BrutalBirdieB Offline
    BrutalBirdieB Offline
    BrutalBirdie Staff
    wrote on last edited by BrutalBirdie
    #5

    The setups have a mixed setup of domains actually.
    3-4 Domains with manual DNS config and 1x domain with DigitalOcean.
    And the processed records in the top log are from the main Cloudron domain which is manual DNS.
    So its even more strange that he fails there with DO when its not even used yet.


    MISTAKE! Was locking at the wrong tab 😄 😬

    So the instance has one 1x Domain with DigitalOcean.
    The DO Team uses nothing but the DNS feature.
    Only one token active.

    So this really does not make much sense.


    System Information:
    Cloudron: v7.4.3
    Ubuntu 22.04

    Like my work? Consider donating a drink. Cheers!

    1 Reply Last reply
    0
  • BrutalBirdieB Offline
    BrutalBirdieB Offline
    BrutalBirdie Staff
    wrote on last edited by
    #6

    Opened a support ticket with DO since this makes no sense at all.

    Like my work? Consider donating a drink. Cheers!

    1 Reply Last reply
    1
  • BrutalBirdieB Offline
    BrutalBirdieB Offline
    BrutalBirdie Staff
    wrote on last edited by
    #7

    The support simply shrugged it of as "check on your side that you don't do more then 250 requests a minute and then it should work".

    I am drilling a bit.

    Like my work? Consider donating a drink. Cheers!

    1 Reply Last reply
    0
  • BrutalBirdieB Offline
    BrutalBirdieB Offline
    BrutalBirdie Staff
    wrote on last edited by
    #8

    Update from DO > @staff :

    🤷


    Hello Elias,

    Thank you for your patience.

    We reviewed the backend logs and noticed that when the requests are sent to "/v2/domains/DOMAIN.TLD/records" and rate limit message shows up "rate_limiter_burst_blocked", however, I am not able to exactly verify the number of requests your API token is making per minute.

    image.png

    I am checking this with our internal team, but in the meantime, I would request you to set up exponential back-off or delay between requests to avoid hitting these limits and flooding the API with requests. If your requests can be spaced out over multiple minutes, you shouldn't hit the limit.

    For more information and insights, check out this article here:

    https://dzone.com/articles/understanding-retry-pattern-with-exponential-back
    

    Here is an example that lists a few scripts on how you can implement an exponential backoff:

    https://cloud.google.com/iot/docs/how-tos/exponential-backoff
    

    In the meantime, if you have any additional questions, please do not hesitate to ask. We're always happy to help whenever we can.

    Like my work? Consider donating a drink. Cheers!

    girishG 1 Reply Last reply
    1
  • girishG Do not disturb
    girishG Do not disturb
    girish Staff
    replied to BrutalBirdie on last edited by
    #9

    @BrutalBirdie how many subdomains/records are there in that domain ?

    1 Reply Last reply
    0

  • Login

  • Don't have an account? Register

  • Login or register to search.
  • First post
    Last post
0
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks