@tparker I tested what @nebulon suggested on my own WP "developer" website and uploading a custom folder with images works fine. The uploaded folder has "root" permission, so make sure to change that to "www-data".
By the way, I can still use WP's auto-update feature for plugins so it's not completely "unmanaged".