Circling back after some testing.
Rsync with backblaze didn't seem to perform very well. Tho i'm not sure i had it configured correctly. It looked like it tried to sync everything each time.
Rsync set up with hetzner storage box works like a charm. Each sync after the first one is fast.
I found an solution to this issue by switching to rsync for backups (with hetzner storage box).
Since each backup is incremental, it doesn't matter how many backups are stored.
The backup size pretty much matches the app size.