<![CDATA[Please randomize time-of-day for certificate renewal]]>I work at Let's Encrypt, which Cloudron uses to issue certificates per its documentation. We've identified that Cloudron disproportionately sends traffic at midnight and noon UTC. I suspect this is the automated renewal noted in Cloudron's documentation.

Let's Encrypt receives disproportionately high amounts of traffic at midnight UTC in particular, as well as the top of other hours. I'm attempting to find software which hardcodes these times to ask the developers to have them renew at a random time of day.

Would it be possible to have cloudron renew at a random time of day? If so, is there an update mechanism that would allow existing clients to install this fix?

Thank you in advance,

Matthew McPherrin
Let's Encrypt Site Reliability Engineering

]]>https://forum.cloudron.io/topic/7410/please-randomize-time-of-day-for-certificate-renewalRSS for NodeWed, 07 Oct 2026 21:23:22 GMTThu, 21 Jul 2022 23:31:41 GMT60<![CDATA[Reply to Please randomize time-of-day for certificate renewal on Mon, 25 Jul 2022 13:50:56 GMT]]>Alright with https://git.cloudron.io/cloudron/box/-/commit/5b4a1e0ec12554f7a4f1099bfc94d041abfca5f5 we now generate a random hour per Cloudron which acts as a seed to spread the instances over 24h.

]]>https://forum.cloudron.io/post/51242https://forum.cloudron.io/post/51242Mon, 25 Jul 2022 13:50:56 GMT<![CDATA[Reply to Please randomize time-of-day for certificate renewal on Sun, 24 Jul 2022 22:42:07 GMT]]>The peak of the spike is about 20 seconds long, where we are returning some errors due to overload. Traffic is elevated for a few more minutes.

Tracking down the spikes is a little fun, though sometimes quite tricky. Cloudron was straightforward because it supplies a distinct user agent, which not all clients do.

]]>https://forum.cloudron.io/post/51227https://forum.cloudron.io/post/51227Sun, 24 Jul 2022 22:42:07 GMT<![CDATA[Reply to Please randomize time-of-day for certificate renewal on Sat, 23 Jul 2022 03:31:30 GMT]]>@mcpherrinm How long does the midnight spike last?

How do you enjoy tracking these down? 🙂

]]>https://forum.cloudron.io/post/51196https://forum.cloudron.io/post/51196Sat, 23 Jul 2022 03:31:30 GMT<![CDATA[Reply to Please randomize time-of-day for certificate renewal on Fri, 22 Jul 2022 23:11:14 GMT]]>Thank you for promptly addressing this!

It’s not just cloudron but many pieces of software all making the same decision to renew at midnight, which leads to uneven traffic.

]]>https://forum.cloudron.io/post/51195https://forum.cloudron.io/post/51195Fri, 22 Jul 2022 23:11:14 GMT<![CDATA[Reply to Please randomize time-of-day for certificate renewal on Fri, 22 Jul 2022 17:00:36 GMT]]>Hi @mcpherrinm thanks for bringing this up. I guess we haven't thought Cloudron became that popular to spread load a bit over time. Currently you can see the hardcoded cron schedule for cert renewal checker at https://git.cloudron.io/cloudron/box/-/blob/master/src/cron.js#L118

We will make this randomized for the next release then.

]]>https://forum.cloudron.io/post/51189https://forum.cloudron.io/post/51189Fri, 22 Jul 2022 17:00:36 GMT