To update everyone here, this is fixed with https://git.cloudron.io/platform/box/-/commit/21191bdc5075bfd334c11d9ffc321e222fa0b6d4
Turns out the code would cleanup ssh identity files when a temporary mount (for the app import) gets removed, despite that key file still being used in another volume or backup site. Root cause was that the key file name was just the host of the remote, which is not unique. We have made the filenames now unique.