Preferred Chain option when renewing certificates

[lcd_official]

Hi! I am submitting this here because it’s not really a bug - at least not for Cloudron.

I have encountered the same situation with Android clients in my implementation of AdGuard Plus that is described here: Let’s Encrypt and DNS over TLS on Android.

Long story short, Let’s Encrypt’s expired X3 certificate causes problems for Android clients on DNS-over-TLS.

The issue can be mitigated by invoking the option —preferred-chain “ISRG Root X1” when renewing Let’s Encrypt certificates, which excludes the expired certificate from the cert chain.

It's in no way a Cloudron bug, but since Cloudron handles certificate renewal, it seems to me to be the best place to insert a solution.

I believe Cloudron uses the ACME API in order to request renewals from Let’s Encrypt, and I would love to have the ability to specify the above option (or perhaps the ability to request other options as well might also be useful to others) within the Cloudron UI or API as part of the certificate setup/renewal.

Thanks for your time, and please let me know if you need any clarification at all.

Thanks!

[girish]

@lcd_official thanks for reporting! I did also very much like your investigation post .

We have our own implementation of acme and we don't use certbot - https://git.cloudron.io/cloudron/box/-/blob/master/src/acme2.js . BTW, if it helps for future investigations, our code is source available.

[lcd_official]

@girish Awesome! Thanks for sharing the name of the code file. I will look through and see if there is any way to implement something similar to certbot's '--preferred-chain' option and will share here if I can find any workaround.

Thanks!

[girish]

@lcd_official Oh, I forgot to update here, but I investigated over the weekend and found the issue.

(Your probably know this, but I am just writing for general info).

Let's Encrypt issues certs signed with R3 intermediate (which is Signed by ISRG Root X1). For backward compat with old android devices, there is also a X1 cross signed with IdenTrust's DST Root CA X3”.

When we request the certificate "chain" via LE API, it gives us both the above two certs - R3 and the cross-signed one. It is possible to request an "alternate" chain via the LE API which only gives the first cert.

It seems that Android's DNS over TLS has a (mis)feature that it gets tripped by the fact that X3 has now expired. Apparently, expiry of X3 does not affect older Android devices (https/browser related logic).

I think doing this at a platform level is probably not needed. We can simply fix AdGuardHome package to simply edit the cert exposed in /etc/certs/tls_cert.pem to not have the final cross signed cert. What do you think?

[lcd_official]

@girish Yes, that is exactly right. That matches with what I found as well.

Sure, I think that would make sense, if the AdGuardHome package could remove the X3 cert at start - perhaps going as far as making it an option exposed via checkbox for the user to include the X3 cert to support "legacy" Android devices, based on preference...

Ultimately I believe this to be an Android issue, in the way that they have rigidly implemented DNS over TLS, but I doubt a change to Android will be anywhere near a quick fix to get implemented

Let me know your thoughts...

Thanks again

[girish]

@lcd_official I have tried to fix this in the latest package. Can you upgrade AdGuard and try?

[lcd_official]

@girish Hi!
Sorry for the late response…. Last few days have been busy…
As of package version 1.4.6 it seems it may be working, at least surviving a reboot. I will perform additional testing in the morning including a manual renew of the cert and will report back asap.

Thanks!

[lcd_official]

@girish Good morning!
It appears to be working perfectly! Tested renewing cert and Android’s Private DNS function is still happy to connect to AGH via DoT. That’s amazing - thank you for implementing that so quickly!

I have to say, after using Cloudron for a while now, what you guys have built here is nothing short of amazing - with so much automation - app installation, VPS security patches, managing certs, auto DNS config + providing LDAP, quick access to Cron, integration with registrars, custom apps - I can go on for days.

This really is bringing self-host to the masses…

Thank you so much

[girish]

@lcd_official thanks for the kind words