The radicale ldap plugin only does authentication. So, in fact, there is no way for radicale to know about any CN. I suspect this is being generated by something else (afaik, the client generates the events file). In my testing, organizer looks like below:
ORGANIZER;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:girish@cloudron
.io
Reply to also works correctly.