Move Navidrome cache out of /app/data so it isn’t backed up

[jdaviescoates]

I’ve been looking into why my rsync backups have got so slow, and file count turned out to be the main factor.

In my Navidrome app, /app/data/data/cache has about 68,000 files, 71% of the app’s total.

It’s all resized artwork and transcodes that Navidrome regenerates on demand, so there’s no value in backing it up, but with rsync to an sshfs target every one of those files costs time on every run.

Could the cache be moved to a persistent dir outside /app/data, as was done for Mastodon?

Navidrome supports setting the location with ND_CACHEFOLDER, so it should be a simple change.

Happy to test.

Thanks.

[timconsidine]

@jdaviescoates I thought rsync only copies what has actually been changed, but still good point (maybe navidrome is aggressively regenerating).

[jdaviescoates]

@timconsidine said:

only copies what has actually been changed

It does. But it still has to check every file to see what has changed and that's what takes the extra time when you've got tens of thousands of files that don't need to be backed up in the first place.

[jdaviescoates]

jdaviescoates said:

@timconsidine said:

only copies what has actually been changed

It does. But it still has to check every file to see what has changed and that's what takes the extra time when you've got tens of thousands of files that don't need to be backed up in the first place.

But it turns out file count wasn’t the main problem!

It was mostly two huge database dumps. Fider had an 11 GB logs table (a 15.6 GB dump), left over from older package versions, as described in this thread.

And Matrix had 86 million rows of state for an old room created by @abuse:matrix.org that nobody on my server is even in any more.

Clearing those cut my rsync backup from nearly 3 hours to under 10 minutes!

Moving the Navidrome cache out of /app/data would still be good, but it’s clearly minor by comparison.

[timconsidine]

Glad you got it resolved.