Storage management in Immich?

[nebulon]

Not sure if immich is the right tool for this then. Anyways though, it only has one data root, so if this is some mounted (network) disk, the upload button would also upload there.

All in all though, Cloudron is designed to back up apps as whole, random things may happen if database and assets on disk go out of sync due to a restore. Not sure if we can realistically support such situations, as we have to find a line between the various use-cases and thus have to make some opinionated decisions.

[shrey]

What i've ended up implementing so far:

• move immich data to an external (mounted) volume
• disable automatic backups
• maintain automatic snapshotting of the volume, externally

Just need to figure out how to maintain backups of the db, externally.
And then, hope that it all works out if the need to restore everything arises.

[nebulon]

I would not recommend implementing a custom backup scheme, easy to miss things. If you use some sshfs rsync backup storage with hardlinks, the backups would be quite efficient. For extra safety the snapshotting of course helps, but anything custom will cause more trouble if things go wrong in the future.

[andreasdueren]

How are people managing this? I am interested in setting Immich up for something but that would require about 10TB of storage. Really only an option with Hetzner Storage Box. I don’t care too much about a backup of the application. And I can set up backups of the photo files from the storage box.

[jdaviescoates]

I don't use Immich but can you not just mount a Hetzner Storage Box as a Cloudron Volume and then mount that as app storage? Volumes don't get backed up. That's what I do with all my Music. The originals are on my laptop. They are synced to a Hetzner Storage Box setup as a Cloudron volume and then that volume is mounted into Navidrome and Jellyfin. Presumably something similar could be done with Immich?

[andreasdueren]

@jdaviescoates I'm not seeing how I can tell Immich to store the files on the external drive. For that I'd need modify the .env file.

[nebulon]

Maybe the external library feature would work with Immich as mentioned in https://forum.cloudron.io/topic/13587/mount-nextcloud-data-as-immich-external-library

[andreasdueren]

@nebulon I've looked into it but from what I understand these are read only from the APP. I'd like to use the app for uploading, not just viewing.

[perelin]

Same use case here: would like to manage my photos with Immich, but want/need to store them on a Hetzner Storage Box (or any other external storage). My library is just to big to put it on the actual Cloudron Server. How can this be achieved? Did not find any clear path to this. Storage Templates maybe?

[jdaviescoates]

Could we not just use the config settings shown here to point at mounted Cloudron Volume? https://immich.app/docs/guides/custom-locations/

[andreasdueren]

@jdaviescoates nope, since we don't have access to the dotenv file

[jdaviescoates]

@andreasdueren said in Storage management in Immich?:

@jdaviescoates nope, since we don't have access to the dotenv file

Sounds to my like @staff just need to add a symlinked .env file to the package that we can add config options too using the File Manager, just like many other packages already have?

[joseph]

I think the immich approach is to set thing in /app/data/immich.json

[perelin]

@joseph the immich.json does not expose a config setting for the file upload/storage path [1]. Looks like the way to go is the env var UPLOAD_LOCATION [2]

[1] https://immich.app/docs/install/config-file/
[2] https://immich.app/docs/install/environment-variables/#docker-compose

[nebulon]

So those env variables are for the immich docker compose environment, which Cloudron does not use. So they won't help us here. The config file does not have this, since it is not something which should be configured.

Instead like the docker compose setup, our package needs to be able to deal with paths accordingly, so we would have to essentially introduce our own "env" vars to define paths to be able to use Cloudron volume mounts. However given that those are not backed up in the same way main app storage is, a restore will inevitably cause issues with the files in storage getting out of sync with the app's database.

All in all I am really not sure if we should even lead users down that path with the risk involved for the benefit of cheaper storage

[perelin]

To understand the issue: when I restore a backup there might possibly be

a) new photos in the external store that are not in the restored db because they were added to Immich in inbetween backup and restore time

and/or

b) photos missing in the external store that are still in the restored db because they have been deleted from Immich inbetween backup and restore time

Both sound like states that Immich should address at some point in their ongoing development. Either re-index the storage path or clean db of file entries no longer available. But I have not checked if they have discussions ongoing about such features.

I generally would opt for letting users shoot themselves in the foot, given enough red, bold disclaimers

[jdaviescoates]

Yeah, I mean, I don't see how Immich is really any different to Calibre Web, Navidrome, Jellyfin etc etc that all allow for custom paths for the location of their libraries and just routinely reindex that location occasionally

[perelin]

@nebulon are you open to explore this further at some point in the future? Immich seems pretty pointless (to me) without the option to plug-in your own storage.

[jdaviescoates]

@perelin I think best bet is to relay that upstream directly to Immich

[perelin]

@jdaviescoates Hard to make a case there when all of their installation methods are somewhat around Docker Compose or similar systems.

From their Docker Compose file I can see that they just mount an external folder to their internal, hardwired upload location -> https://github.com/immich-app/immich/releases/download/v1.132.3/docker-compose.yml

I dont know where I can lookup the Cloudron/Immich integration, but I guess the mounting could happen when the Dockerfile gets run. But of course at first there is no mounted file system available