Invalid response code when fetching directory : 429

[d19dotca]

@girish Yeah totally fair enough. It's a Let's Encrypt issue that we're comfortable will resolve itself (and it seems that's the behaviour we've seen over time too) then maybe the only change needed in Cloudron is to make it less aggressive. Perhaps start renewing 30 days in advance as it does already, but don't notify of any problems until maybe 15 or 20 days in advance? That way presumably it'd be an issue that's occurring for roughly 10 days prior repeatedly that wasn't yet resolved if we see any notifications like that? Hopefully that makes sense.

I like your idea, I think less aggressive notifications for issues we know tend to resolve themselves over time would be helpful. And more importantly it'll save a bit of time on your end in less posts haha

[d19dotca]

@girish Just found this too: https://bobcares.com/blog/lets-encrypt-new-auth-status-429-error/ - Seems like one possible cause of this is too many subdomains in Cloudron for a single domain, and Let's Encrypt limiting how many are given out for the domain each week. In which case less aggressive notifications is probably a great change to be made.

I wonder if a second improvement here could be to show the actual Let's Encrypt response in its totality, rather than stripped down in Cloudron? That may be helpful so people at least know if the issue is in Cloudron or coming from Let's Encrypt's side.

[girish]

@d19dotca On Cloudron, this seems to happen when querying the public URL - https://acme-v02.api.letsencrypt.org/directory . That links provides a directory map of URLs (instead of hardcoding the URLs it in the code base). For some reason that returns a 429. So, it's not related to cert limits or account limits.

[girish]

If i see https://tools.ietf.org/html/rfc8555#page-23, there is no 429 response code.

[robi]

I am getting this message on a few domains as well.

[girish]

Does curl https://acme-v02.api.letsencrypt.org/directory return an error on the server?

[robi]

@girish no, all come back with data.

What about setting up a time to run these curls around the same time cloudron does (and log it), which may be when LE does something on their systems.

Also shifting the time cloudron does it may be good. There's hundreds of cloudrons hammering them at midnight for even more domains

[girish]

For lack of ideas, I have added a retry now when fetching directory.

[imc67]

@girish on one Cloudron I already get for the third time in a row (so each time after 12 hours) these below errors, usually it was only one time. Since this moring (so after the second and before the third) I updated Cloudron to 6.2.7:

The error was:

-------------------------------------

Invalid response code when fetching nonce : 429

-------------------------------------


Powered by https://cloudron.io

Sent at: Mon, 29 Mar 2021 12:00:07 GMT

The error was:

-------------------------------------

Unexpected status: invalid

-------------------------------------


Powered by https://cloudron.io

Sent at: Tue, 30 Mar 2021 00:05:38 GMT

The error was:

-------------------------------------

Invalid response code when fetching nonce : 429

-------------------------------------


Powered by https://cloudron.io

Sent at: Tue, 30 Mar 2021 12:00:09 GMT

[girish]

@imc67 in the next release, we will suppress those notifications. it seems to not add much value. in any case, the solution is just to wait since cloudron will try again later. it's not a Cloudron issue, the LE service just goes up/down by nature.