Wrong directory for previewcleanup cron job

[tobiasb]

The cron job previewcleanup with the script /app/pkg/preview-cleanup.sh tries to change to non-existing directory.

cd /app/data/data/appdata_${instanceId}/preview/

should be

cd /app/data/appdata_${instanceId}/preview/

[james]

Hello @tobiasb
Thanks for reporting.
In addition to this, I also think that preview images should not really be in /app/data/.
They are not really worth being included in the backup since they can be regenerated at any time.
With the new Cloudron versions there is now persistentDirs so we should move them outside of /app/data/.

[james]

MR: https://git.cloudron.io/packages/nextcloud-app/-/merge_requests/61/diffs

[tobiasb]

Oh I see, that new installation get /app/data/data. So the script is only buggy for the old installation. But yes move the previews out of backups.

[avatar1024]

@james said:

In addition to this, I also think that preview images should not really be in /app/data/.
They are not really worth being included in the backup since they can be regenerated at any time.
With the new Cloudron versions there is now persistentDirs so we should move them outside of /app/data/.

Yes please! NC previews can be a nightmare with rsync backups because it adds so many small files.

[james]

Hello @tobiasb and @avatar1024
A Nextcloud update will we available tomorrow morning that fixes the issue and moves the image previews out of /app/data/

[msbt]

@james that update took down one of my Nextcloud instances. It was offline for 6 hours, stuck at ==> Resetting previews. There are maybe 300GB of images on that instance and it never managed to finish the cleanup (I think), only after I manually created /var/lib/nextcloud/preview it would come back up again. Not sure if the cleanup script was done that very minute or if there was an issue regarding that directory.

[james]

Hello @msbt
Thanks for the report, I will see if I can replicate that issue.

[james]

Hello @msbt

Is your Nextcloud on an HDD or maybe a mounted network volume (NFS, CIFS)?
It could be that because of a slow disk this really did take this long.

I did some measurements, inspected Nextcloud code, and issues.
There are many issues on GitHub complaining about image previews https://github.com/nextcloud/server/issues?q=is%3Aissue+state%3Aopen+preview+empty+folders

By default, Nextcloud creates 4x image previews for each image.
At an assumed 3-5 MB per photo, 300 GB is roughly 60k-100k images. The file size is an assumption.
That means roughly 240k-400k preview rows and files, and about 55-95 GB of previews.

Rows? Yes, Nextcloud stores each preview file in the oc_previews PostgreSQL table.
And the command that we currently use in the start.sh is $occ preview:cleanup.
This command goes through every preview image synchronously, one by one, deletes the file and then the row in oc_previews.

Then there is another issue that empty preview folders are never cleaned up by Nextcloud.
We already did implement the cron that this topic is about that cleans up these empty folders.
Also, multiple GitHub issues dating back to 2024 and still this was never fixed upstream.

That $occ preview:cleanup is working synchronously is in my opinion extremely inefficient.
The command is for deleting all preview images, so why do it one by one?
If the command is for deleting all, why not simply delete all at once and truncate the oc_previews table?
I don't know why they don't do it that way.

I will create an update for the Nextcloud app that moves this process outside the start.sh so the Nextcloud can start and that clean-up will be done in the background and if my idea of delete all at once and truncate the oc_previews is faster and does not cause issues will use that instead of the slow $occ preview:cleanup.

[msbt]

@james that would have been my theory as well, because my data folder is a Hetzner Storagebox mounted via sshfs. If it's outside of start.sh it shouldn't be a problem, because the instance would start successfully and can then do the things needed. Appreciate the efforts!