Rsync backup integrity fails for one Nextcloud and Jellyfin.

[jdaviescoates]

@james I'd like to get to the bottom of why my rsync Nextcloud and Jellyfin backups always fail the integrity check.

I'm pretty certain it's not something happen during the backup process because if that were the case it'd only happen on some rare occasions when I'm active during the backup process, but it is literally every single rsync back up with both of these apps.

Nextcloud always seems to say:

Failed to verify 2026-05-31-000001-449/app_office.uniteddiversity.coop_v5.8.2: Cannot read properties of undefined (reading 'size')

and

Jellyfin similarly always seems to say:

Failed to verify 2026-05-31-000001-449/app_media.uniteddiversity.coop_v1.13.7: Cannot read properties of undefined (reading 'size')

So both seem to be having the same issue:

Cannot read properties of undefined (reading 'size')

It doesn't happen on my other Nextcloud instance.

Any idea how we might get to the bottom of it?

BTW: out of interest I figured I'd try doing what you suggested previously (even though I don't really see how that'd help us work out what's going on) I tried to put the Nextcloud into maintenance mode using this command as per https://docs.nextcloud.com/server/stable/admin_manual/maintenance/backup.html#maintenance-mode

$ sudo -E -u www-data php occ maintenance:mode --on

But that just returned:

bash: $: command not found

[james]

Hello @jdaviescoates
The ooc tool is called this way https://docs.cloudron.io/packages/nextcloud#running-occ-tool

sudo -u www-data php -f /app/code/occ $COMMAND

[jdaviescoates]

@james said:

Hello @jdaviescoates
The ooc tool is called this way https://docs.cloudron.io/packages/nextcloud#running-occ-tool

sudo -u www-data php -f /app/code/occ $COMMAND

Thanks, that worked.

[jdaviescoates]

@staff I've now got to the bottom of this with the help of Claude (who also wrote most of this post).

It looks like potentially a genuine post-Cloudron-migration rsync backup bug rather than anything to do with files changing during the backup.

Summary

My rsync backups of Nextcloud and Jellyfin fail the integrity check every single night. Tarball backups of the same apps always pass. On 9.2 the error is:

Failed to verify 2026-09-22-000001-401/app_office.domain.coop_v5.8.8: Cannot read properties of undefined (reading 'size')

The backup task itself reports no errors.

What I found

I compared a full file listing of each app's live data with the corresponding folder in the rsync backup on my Hetzner Storage Box (mounted via SSHFS).

Jellyfin: the backup holds 2,983 files that no longer exist on the server. These are old People and artist metadata images plus an old plugin folder (plugins/SSO Authentication_4.0.0.4). Deleted files are never removed from the backup. Going the other way, 18 files that have existed on the server since 15 March have never been uploaded.

Nextcloud: the backup holds 416 files that no longer exist on the server, mostly JS and CSS from old versions of Nextcloud apps. More seriously, real user files are missing. 21 of the 28 files in one of my folders are not in the backup, along with 5 files in another folder. They have been on the server for months.

So the rsync backup is both keeping deleted files and missing files that exist. My tarballs contain everything, so nothing is lost, but the rsync backups could not be relied on for a restore.

The likely cause

I did a full Cloudron migration onto a new server freshly built on 13 March (Cloudron installed that evening). The files that never uploaded, in both apps, have modification times in the early hours of 15 March, many at exactly 02:11. After the migration, the new server carried on backing up to the same rsync folder the old server had used.

My guess is that the new server's record of what it had already uploaded never matched what was actually on the Storage Box. Files it thinks are uploaded are skipped every night, and files left over from the old server are never deleted because it doesn't know about them. That would explain why it fails every time, only in rsync format, and only for these apps.

In April, on 9.1.5, the same problem showed up as a file count mismatch (284,709 found vs 285,450 expected). On 9.2 the checker seems to hit the mismatch and crash instead.

Questions

1. How does the rsync format track what has already been uploaded, and is there a safe way to force a full resync for specific apps?
2. Should a migration onto an existing rsync target reset or rebuild that state?
3. Could the integrity checker list the missing or unexpected files instead of throwing? It would have made this much quicker to diagnose.

In the meantime I'm planning to add a fresh rsync target in an empty folder, check it passes, and then remove the old one.

[james]

Hello @jdaviescoates
Interesting finding.
I did not try to reproduce this yet, but this already reads like it could be hard to reproduce.

Maybe you could do a test if possible.
If you add a new storage site, that is also rsync and a hetzner storage box and create a fresh backup maybe this issue does not exist on the new backup site then.

Since you have already indentified single files that are not deleted and not uploaded it might be good to share this information with us so we can have a look on your system to see what is actually happening.
If that is possible, please write a mail to support@cloudron.io with the details and enable remote support on your Cloudron server.

[jdaviescoates]

@james said:

Maybe you could do a test if possible.
If you add a new storage site, that is also rsync and a hetzner storage box and create a fresh backup maybe this issue does not exist on the new backup site then.

That's what I'm doing at present... will report back once it finishes and I've done the integrity check...

[jdaviescoates]

jdaviescoates said:

That's what I'm doing at present... will report back once it finishes and I've done the integrity check...

Well, it failed again. But it wasn't the persistently failing instances of Nextcloud and Jellyfin this time, but the mail server instead. So, progress!

[girish]

It's possible that rsync backups from the past (before we introduced integrity checks) are not backed up properly or are missing integrity information. If you can do a full backup from scratch (like you did for nc,jellyfin) and if it's reproducible, then that would be a sure bug.

[jdaviescoates]

@girish I think it was just the mismatch caused by migrating the server that led to NC and JF persistently failing. My guess is mail will be ok next time too... and that that may well have been caused just by how long it took to do this new full rsync backup and integrity check

[jdaviescoates]

jdaviescoates said:

My guess is mail will be ok next time too

Yep, the most recent rsync back-up into the new folder (rather than the same folder as the rsync backups were going into before the server migration) worked fine too and now I have rsync backups that pass the integrity check - finally!

So it seems clear the issue was some file changes that happening during the full Cloudron server migration got rsync confused.