Changing calendars on an event via Calendar app on macOS and iOS doesn't work

[d19dotca]

@nebulon Yes, same Radicale instance for all my calendars. And I just used the built-in Calendar app that comes with macOS.

[d19dotca]

@nebulon Quick update to say I tried on the latest Radicale update you deployed earlier today, but same thing. 502 Bad Gateway error upon MOVE commands.

[nebulon]

Ok got access to a macbook, however it works for me:

18:47:20 - [2020-09-15 16:47:20 +0000] [1/Thread-2416] [INFO] GET response status for '/' in 0.000 seconds: 302 Found
18:47:26 - [2020-09-15 16:47:26 +0000] [1/Thread-2417] [INFO] MOVE request for '/nebulon/7605fbf0-7bd3-63a0-c8df-e5c4af4f34fd/f9d03e14-303a-4f5d-8c8e-eeaf4cb33690.ics' received from '90.187.246.49' (forwarded by 172.18.0.1) using 'Mac+OS+X/10.14.6 (18G103) CalendarAgent/416.5.1'
18:47:26 - [2020-09-15 16:47:26 +0000] [1/Thread-2417] [INFO] Successful login: 'nebulon'
18:47:26 - [2020-09-15 16:47:26 +0000] [1/Thread-2417] [INFO] MOVE response status for '/nebulon/7605fbf0-7bd3-63a0-c8df-e5c4af4f34fd/f9d03e14-303a-4f5d-8c8e-eeaf4cb33690.ics' in 0.192 seconds: 201 Created
18:47:26 - [2020-09-15 16:47:26 +0000] [1/Thread-2418] [INFO] PROPFIND request for '/nebulon/f1ce483f-5dac-87e1-d2c7-e4706444faab/f9d03e14-303a-4f5d-8c8e-eeaf4cb33690.ics' with depth '0' received from '90.187.246.49' (forwarded by 172.18.0.1) using 'Mac+OS+X/10.14.6 (18G103) CalendarAgent/416.5.1'
18:47:26 - [2020-09-15 16:47:26 +0000] [1/Thread-2418] [INFO] Successful login: 'nebulon'
18:47:26 - [2020-09-15 16:47:26 +0000] [1/Thread-2418] [INFO] PROPFIND response status for '/nebulon/f1ce483f-5dac-87e1-d2c7-e4706444faab/f9d03e14-303a-4f5d-8c8e-eeaf4cb33690.ics' with depth '0' in 0.180 seconds: 207 Multi-Status

Are you using calendars of different users?

[nebulon]

Ok I also tried with different accounts and then the macos client also performs a DELETE+ADD instead of a MOVE, so not sure how to debug this further for me

[girish]

The log message of "Unsupported destination address" comes from https://github.com/Kozea/Radicale/blob/7ed51226360212d2958a142d7d193b3bdd0c0e40/radicale/app/move.py#L34 The code there says to_url.netloc != environ["HTTP_HOST"]. From a casual reading, could it be that the domain of the calendar is hardcoded somewhere inside radicale? Meaning, if the user moved the radicale app from one location to another, maybe some internal entries are not patched up?

This happens for SOGo which I discovered just a couple of days back - https://git.cloudron.io/cloudron/sogo-app/-/issues/19

[d19dotca]

@nebulon & @girish - very interesting. According to the Radicale GitHub it's been an issue for others before and sometimes fixed and sometimes needing the proxy to be modified.

One thing worth mentioning in case this is an issue, though it is released now (or will be very soon so this will be an issue down the road perhaps) is I was running macOS 11 (i.e. the latest one released for September 2020 called Big Sur, running on their public beta builds earlier). So maybe there's something different now in how their Calendar app handles changes to calendars - no longer doing a DELETE+ADD but a MOVE now - although it's odd because it wasn't an issue earlier in the beta builds and I think either way they're still doing it correctly (just maybe different) as a MOVE method is a valid command to Radicale. Though this may explain why it works for you and not me, as we're running different macOS versions.

You may be on to something though about that part of the code in their GitHub project with "to_url.netloc != environ["HTTP_HOST"]". Never seen that before and it's never been an issue before to my knowledge. Any ideas where that might be stored though? Is that something I need to change in my environment or is that more of an app thing? I was able to reproduce it before on a brand new Radicale deployment, so I assume this isn't something I have control over?

[d19dotca]

Seems the HTTP_HOST value is set too in the proxy according to the docs: https://github.com/Kozea/Radicale/blob/9d25cc6c0ae6fa72bd6c57b76cbd25f1fddedbc0/DOCUMENTATION.md
I assume it's using that set in the nginx reverse proxy in it's comparison logic for the MOVE command? Though still doesn't necessarily explain why it works for you guys though, as I assume that same value would be set anyways, it's the hostname of the app after all, isn't it?

[nebulon]

I don't think the macOS version is the issue here, since I also see the MOVE command being used (but in my case successfully) if I move events between calendars of the same user.

In your case, did you ever move that radicale app instance from one domain/subdomain to another?

[d19dotca]

@nebulon I don't believe I have, I think I've added additional domains though. Basically the subdomain for the Radicale instance is "dav" and to my knowledge always has been. However more recently I added in a couple of additional locations like "calendars" and "contacts", etc.

I think what I'll try next is deploy a fresh Radicale instance but specifically only with one location, and making sure the Calendar app uses that one alone and see if I can reproduce it. I had tried before but I think I had also added in additional domains again, so I'll try again with just one, because maybe that's related to the host name check in the Radicale code?

[d19dotca]

@nebulon & @girish - Okay, great call on the hostname thing, that was indeed the issue for the 502 Gateway errors. So I think what happened there was I originally had everything on dav.<domain> and that's when I was getting the 409 Conflict messages originally reported back on August 17th. But then in troubleshooting I tried migrating everything to a new Radicale instance and at that time decided to also use some extra locations such as calendars.<domain> and contacts.<domain> to make things a little more user friendly. Turns out that was what caused the 502 Bad Gateway.

Now that I re-added my account in Calendars app on macOS to be only looking at the original dav.<domain>, I am back at square one with the 409 Conflict error in the logs.

Sep 17 09:53:01 [2020-09-17 16:53:01 +0000] [1/Thread-7226] [INFO] MOVE response status for '/<username>/<CalendarName>/2BEA501D-3AFF-4DAB-822E-CA90AD276E08.ics' in 0.219 seconds: 409 Conflict

Any ideas? Unfortunately we're sort of back at square one still, as I can't migrate from one calendar to another.

I think my next task will be to create a new Radicale instance then with only one address, then start creating some new entries from scratch and see if the issue still occurs or not. If not, then I'll try to migrate in my ics exports and see if the issue then comes back, because if it does then perhaps that may point to some possible corruption in the data I have? I'll report back when I test this out.

[d19dotca]

Another update: Quickly spun up a new Radicale instance at davtest.<domain> and created a new event and then moved it to another calendar on the same instance, and now I get it successfully moved.

Sep 17 10:03:33 [2020-09-17 17:03:33 +0000] [1/Thread-124] [INFO] MOVE response status for '/<username>/87F80099-3192-4C3C-A7E4-C64B864557B1/2BEA501D-3AFF-4DAB-822E-CA90AD276E08.ics' in 0.207 seconds: 201 Created

Next I will try moving in the data from the other calendars from the original Radicale instance via exporting the calendar as an ics file via the macOS Calendar app, import to the new instance, and see if I can then still move events to different calendars.

[d19dotca]

Okay, it seems it's working now with a new instance of Radicale. No idea why the 409 Conflict happened in the first place though. Here's what I have learned, for anyone who comes across this in the future:

• It seems that Radicale does not like multiple hostnames / redirects to the main hostname, and clients connecting to the redirect name causes the 502 Bad Gateway errors.

• If you run into the 409 Conflict, I guess just migrate everything to a new instance, and it somehow resolves it. lol.

Thanks for everyone's help on this!

[girish]

Great stuff, persisting with it!