Radicale sync slow between devices, poor performance in particular with the CalDAV data.

[d19dotca]

I have been finding the sync between devices via Radicale quite slow at times, for a while now but I've sort of just "put up with it", but now am really wanting to see what I can do to improve the performance of Radicale. Unfortunately all of the Radicale configuration is in the read-only /app/code/config file. How might I go about being able to tweak these for testing performance optimizations? In particular I wanted to test with increasing the max_connections value to see if this may help with the synchronization process at all as I do have a lot of data and perhaps more parallel connection threads may help?

Some notable information:

• Platforms used are macOS and iOS, all on the very latest versions. I have even used the beta builds to see if anything new that might fix it assuming it's an app problem client-side, but nothing ever changes in the sync speed.

• I find reminders/tasks sync to be the slowest, but it may be because I'm also using that the most. I would say it takes normally anywheres from 10-45 seconds to sync after opening the app up on iOS after making a bunch of changes to calendar events or contacts or reminders on macOS an hour or two earlier, for example. Usually 15-30 seconds is the typical sync time I see, and it's just quite annoying to wait half a minute before I can really proceed with anymore changes from my mobile device.

• No third-party apps used, only apps used are the default apps in macOS and iOS including Reminders & Calendar apps (for CalDAV), and Contacts app (for CardDAV).

Eventually I want to test also setting up a fresh Radicale instance to see if very little data in it will change anything, in case it's the sheer amount of data in my system. I don't think I'm completely a-typical with it but I do have a lot of contacts (~1600) and I work in Calendar and Reminders a lot, particularly Reminders. So I can imagine the volume of data changed or needing to sync may be a lot (although I'd think only the changes would be synced, not everything each time). Would like to rule this out, will test this eventually.

Additionally I may try to find some third-party CalDAV and CardDAV compatible apps to test with in case the issue is in the built-in apps, but I just doubt that as generally macOS and iOS default apps are pretty compatible with CalDAV and CardDAV, not requiring any additional apps or options.

Any ideas though on the other side of things where I can tweak the /app/code/config file by any chance? I know some apps like Roundcube have that read-only but then allow overwriting certain settings by editing a file in the writable /app/data/ area. Is this something we have in Radicale too?

[nebulon]

I guess in this case the easiest way to do this is to install the cloudron cli tool on your laptop and then put the radicale app instance into debug mode with cloudron debug --app <appid/domain>

Once that is done you can get a remote terminal to the app with cloudron exec --app <appid/domain> and then modify the files in question. Please note that those changes are not persistent! In debug mode, the app is also not running. So after adjusting some config files for testing, run /app/code/start.sh from that terminal session.

Once done with all the testing you can disable debug mode again with cloudron debug --disable --app <appid/domain>

[d19dotca]

@nebulon Okay great, I will try this eventually then. Thanks!

[d19dotca]

I thought I'd update everyone here in case anyone comes across this in the future too. I believe I've found the root cause.

Upon digging into /app/data/collections/collection-root/<username>/<collection_folders>, and running the "du -h" command, I found that there was a lot of data in the "./<collection_folder>/.Radicale.cache/sync-token", some upwards of containing nearly 20 MB of data. According to the Radicale documentation, it can be cleared and it'll be rebuilt. So I cleared it and now everything seems to sync much quicker.

For me, between all my "collections" in Radicale, I had nearly 55 MB of data alone in sync-tokens across the different collections. Since removing that, it's been much quicker at syncing when I make a change on iOS to showing that change on macOS and vice-versa. There isn't a lot of documentation on what the sync tokens are exactly, but since it's sync-related, I would assume this directly impacts the sync process speed if it has to chug through 55 MB of data instead of about 2 MB which is what it was reduced to after clearing them all and having it be rebuilt automatically. Not sure how often I'll have to clear those, but it may be wise to setup a cronjob for such a task. I'll be closely monitoring this over the next while to see how quickly it builds up.

[nebulon]

Great find! I think it would be worth creating an upstream issue with radicale to figure out why those tokens impact performance as well as what the proper way to deal with them is. Blindly deleting them is probably not the ideal case, since that might interrupt ongoing synchronizations.

[d19dotca]

@nebulon Will do. I've checked today and see the sync tokens grew a fair bit again, although right now performance seems to still be okay. So I'll monitor a bit over the next week before filing a ticket just in case it grows back to where it was but still syncs quickly, as that may indicate something related but different as a root cause.