filescan errors
-
Hi there,
When I run "occ files:scan --all" I get quite some errors. They exist since some time, but I had no problems in daylie usage. And searching didnt really get me somewhere.
Does someone have a clue where to start and how to solve them?First it says, for some users, their Folders don't exist. But these users don't have problems.
Path not found: /username
And then there is a bunch of errors, where I really don't know where to start.
Exception during scan: An exception occurred while executing a query: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block #0 /app/code/3rdparty/doctrine/dbal/src/Connection.php(1938): Doctrine\DBAL\Driver\API\PostgreSQL\ExceptionConverter->convert() #1 /app/code/3rdparty/doctrine/dbal/src/Connection.php(1880): Doctrine\DBAL\Connection->handleDriverException() #2 /app/code/3rdparty/doctrine/dbal/src/Connection.php(1105): Doctrine\DBAL\Connection->convertExceptionDuringQuery() #3 /app/code/lib/private/DB/Connection.php(261): Doctrine\DBAL\Connection->executeQuery() #4 /app/code/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(341): OC\DB\Connection->executeQuery() #5 /app/code/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(377): Doctrine\DBAL\Query\QueryBuilder->executeQuery() #6 /app/code/lib/private/DB/QueryBuilder/QueryBuilder.php(280): Doctrine\DBAL\Query\QueryBuilder->execute() #7 /app/code/lib/private/Files/Cache/Cache.php(172): OC\DB\QueryBuilder\QueryBuilder->execute() #8 /app/code/lib/private/Files/Cache/HomeCache.php(57): OC\Files\Cache\Cache->get() #9 /app/code/lib/private/Files/Cache/Cache.php(553): OC\Files\Cache\HomeCache->get() #10 /app/code/lib/private/Files/Cache/Scanner.php(286): OC\Files\Cache\Cache->remove() #11 /app/code/lib/private/Files/Cache/Scanner.php(520): OC\Files\Cache\Scanner->removeFromCache() #12 /app/code/lib/private/Files/Cache/Scanner.php(404): OC\Files\Cache\Scanner->handleChildren() #13 /app/code/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren() #14 /app/code/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren() #15 /app/code/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren() #16 /app/code/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren() #17 /app/code/lib/private/Files/Cache/Scanner.php(354): OC\Files\Cache\Scanner->scanChildren() #18 /app/code/lib/private/Files/Utils/Scanner.php(277): OC\Files\Cache\Scanner->scan() #19 /app/data/apps/files/lib/Command/Scan.php(180): OC\Files\Utils\Scanner->scan() #20 /app/data/apps/files/lib/Command/Scan.php(241): OCA\Files\Command\Scan->scanFiles() #21 /app/code/3rdparty/symfony/console/Command/Command.php(298): OCA\Files\Command\Scan->execute() #22 /app/code/core/Command/Base.php(177): Symfony\Component\Console\Command\Command->run() #23 /app/code/3rdparty/symfony/console/Application.php(1040): OC\Core\Command\Base->run() #24 /app/code/3rdparty/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand() #25 /app/code/3rdparty/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun() #26 /app/code/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run() #27 /app/code/console.php(100): OC\Console\Application->run() #28 /app/code/occ(11): require_once('...') #29 {main} In ExceptionConverter.php line 87: An exception occurred while executing a query: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block In Exception.php line 28: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block In Statement.php line 130: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block
I am running v7.6.2 (Ubuntu 22.04.1 LTS) on a hetzner vserver. Nextcloud version 28.0.5
Any suggestions where to start? -
The folders are in a remote file system mounted via sshfs. When I open FileManger in Cloudron, I don't see a file tree. So no folders at all. But also don't see problems when operating in nextcloud.
When I open that drive with my SFTP-Software, I can see all folders and all show same rights and owner.Back at cloudron file manager, when I open a different Volume, which is a different Folder mounted from the same drive, I can see the containing folders.
During "Path not found: /username" there is nothing happening in the postgres logs.
But when "Exception during scan: ..." appears in the terminal, postgress logs shows this:May 08 10:20:142024-05-08 08:20:14.180 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 ERROR: invalid byte sequence for encoding "UTF8": 0xe4 0x63 0x68 May 08 10:20:142024-05-08 08:20:14.180 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 CONTEXT: unnamed portal parameter $8 May 08 10:20:142024-05-08 08:20:14.180 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 STATEMENT: INSERT INTO "oc_filecache" ("mimepart", "mimetype", "mtime", "size", "etag", "storage_mtime", "permissions", "name", "parent", "checksum", "path_hash", "path", "storage") VALUES($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13) May 08 10:20:142024-05-08 08:20:14.185 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 ERROR: current transaction is aborted, commands ignored until end of transaction block May 08 10:20:142024-05-08 08:20:14.185 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 STATEMENT: INSERT INTO "oc_filecache" ("mimepart", "mimetype", "mtime", "size", "etag", "storage_mtime", "permissions", "name", "parent", "checksum", "path_hash", "path", "storage") VALUES($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13) May 08 10:20:142024-05-08 08:20:14.187 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 ERROR: current transaction is aborted, commands ignored until end of transaction block May 08 10:20:142024-05-08 08:20:14.187 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 STATEMENT: INSERT INTO "oc_filecache" ("mimepart", "mimetype", "mtime", "size", "etag", "storage_mtime", "permissions", "name", "parent", "checksum", "path_hash", "path", "storage") VALUES($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13) May 08 10:20:142024-05-08 08:20:14.189 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 ERROR: current transaction is aborted, commands ignored until end of transaction block May 08 10:20:142024-05-08 08:20:14.189 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 STATEMENT: SELECT "filecache"."fileid", "storage", "path", "path_hash", "filecache"."parent", "filecache"."name", "mimetype", "mimepart", "size", "mtime", "storage_mtime", "encrypted", "etag", "filecache"."permissions", "checksum", "unencrypted_size", "metadata_etag", "creation_time", "upload_time", "meta"."json" AS "meta_json", "meta"."sync_token" AS "meta_sync_token" FROM "oc_filecache" "filecache" LEFT JOIN "oc_filecache_extended" "fe" ON "filecache"."fileid" = "fe"."fileid" LEFT JOIN "oc_files_metadata" "meta" ON "filecache"."fileid" = "meta"."file_id" WHERE ("storage" = $1) AND ("path_hash" = $2) May 08 10:20:142024-05-08 08:20:14.190 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 ERROR: current transaction is aborted, commands ignored until end of transaction block May 08 10:20:142024-05-08 08:20:14.190 UTC [166] user7826f9bec6084e53af12181aa87f33e2@db7826f9bec6084e53af12181aa87f33e2 STATEMENT: SELECT "uid", "displayname", "password" FROM "oc_users" WHERE "uid_lower" = $1
Years ago I migrated this nextcloud from a vserver to a nextcloud within cloudron.
Therefore I converted the database from mysql and imported to nextcloud. Maybe relicts? -
Oh that sshfs adds some interesting angle to the issue. Can you browse the filetree of that sshfs mount via SSH from the server itself without any errors?
If you encounter strange behavior, also check the systemlog for potential kernel errors. Does a remount solve anything in that situation?
-
@nebulon said in filescan errors:
Can you browse the filetree
Yep, no problems here
Nextcloud Log while filescan
May 08 14:42:10- - - [08/May/2024:12:42:10 +0000] "GET /index.php/login HTTP/1.1" 200 7156 "-" "Mozilla (CloudronHealth)" May 08 14:42:1812:M 08 May 2024 12:42:18.008 * 10000 changes in 60 seconds. Saving... May 08 14:42:1812:M 08 May 2024 12:42:18.012 * Background saving started by pid 368 May 08 14:42:19368:C 08 May 2024 12:42:19.389 * DB saved on disk May 08 14:42:19368:C 08 May 2024 12:42:19.391 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB May 08 14:42:1912:M 08 May 2024 12:42:19.415 * Background saving terminated with success May 08 14:42:20- - - [08/May/2024:12:42:20 +0000] "GET /index.php/login HTTP/1.1" 200 7151 "-" "Mozilla (CloudronHealth)" May 08 14:42:30- - - [08/May/2024:12:42:30 +0000] "GET /index.php/login HTTP/1.1" 200 7155 "-" "Mozilla (CloudronHealth)" May 08 14:42:40- - - [08/May/2024:12:42:40 +0000] "GET /index.php/login HTTP/1.1" 200 7155 "-" "Mozilla (CloudronHealth)" May 08 14:42:50- - - [08/May/2024:12:42:50 +0000] "GET /index.php/login HTTP/1.1" 200 7149 "-" "Mozilla (CloudronHealth)" May 08 14:43:00- - - [08/May/2024:12:43:00 +0000] "GET /index.php/login HTTP/1.1" 200 7157 "-" "Mozilla (CloudronHealth)" May 08 14:43:10- - - [08/May/2024:12:43:10 +0000] "GET /index.php/login HTTP/1.1" 200 7155 "-" "Mozilla (CloudronHealth)" May 08 14:43:20- - - [08/May/2024:12:43:20 +0000] "GET /index.php/login HTTP/1.1" 200 7153 "-" "Mozilla (CloudronHealth)" May 08 14:43:30- - - [08/May/2024:12:43:30 +0000] "GET /index.php/login HTTP/1.1" 200 7154 "-" "Mozilla (CloudronHealth)" May 08 14:43:40- - - [08/May/2024:12:43:40 +0000] "GET /index.php/login HTTP/1.1" 200 7155 "-" "Mozilla (CloudronHealth)"
Cloudron Box Log while filescan
May 08 14:42:10box:apphealthmonitor app health: 7 running / 3 stopped / 0 unresponsive ...
Via Cloudron File Manager I opend Nextclouds config.php. At the same time the system log shows this
May 08 14:51:23box:server no such route: GET /api/v1/apps/7826f9be-c608-4e53-af12-181aa87f33e2/files/config/config.phpfiles/app-7826f9be-c608-4e53-af12-181aa87f33e2/config%2Fconfig.php?access_token=<redacted> [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client at new NodeError (node:internal/errors:399:5) at ServerResponse.setHeader (node:_http_outgoing:645:11) at ServerResponse.header (/home/yellowtent/box/node_modules/express/lib/response.js:794:10) at ServerResponse.json (/home/yellowtent/box/node_modules/express/lib/response.js:275:10) at ServerResponse.send (/home/yellowtent/box/node_modules/express/lib/response.js:162:21) at /home/yellowtent/box/node_modules/connect-lastmile/lib/index.js:78:28 at Layer.handle_error (/home/yellowtent/box/node_modules/express/lib/router/layer.js:71:5) at trim_prefix (/home/yellowtent/box/node_modules/express/lib/router/index.js:326:13) at /home/yellowtent/box/node_modules/express/lib/router/index.js:286:9 at Function.process_params (/home/yellowtent/box/node_modules/express/lib/router/index.js:346:12)
-
Strange the URL clearly has the file/api path totally wrong:
/api/v1/apps/7826f9be-c608-4e53-af12-181aa87f33e2/files/config/config.phpfiles/app-7826f9be-c608-4e53-af12-181aa87f33e2/config%2Fconfig.php
Not sure where this is coming from to be honest. All in all it appears there are a bunch of probably unrelated issue actually.
So for nextcloud, you can confirm that using the app itself and downloading/uploading files works just fine, only the file rescanning via
occ
fails? This may or may not be related to sshfs potentially being slow. Not too sure how to debug if ther are no system errors (journalctl --system
ordmesg
). Debugging Nextcloud's php code is a bit out of scope for us.The Cloudron filemanager issue should be fixed though if this is a real problem. Do you have other apps which reside in that sshfs mount point and does the filemanger work there?
-
-
@nebulon said in filescan errors:
So for nextcloud, you can confirm that using the app itself and downloading/uploading files works just fine, only the file rescanning via occ fails?
yep. I had some issues with umlauts when I migrated. But since long there are no file errors while operating.
@nebulon said in filescan errors:
Do you have other apps which reside in that sshfs mount point and does the filemanger work there?
yes, there is another nextcloud app running on a different domain. Dont have files:scan errors there and I see files via FileManager.
@nebulon said in filescan errors:
may or may not be related to sshfs potentially being slow.
I don't see issues there. nextcloud is working fine.
-
@nebulon said in filescan errors:
if you can browse the filesystem via the webterminal into that app instance, does it look correct?
when I go there, I see the local Data Directory of the app, but not the mounted volume content.
This applies for both instance of NextcloudWhen I navigate to /media/volume in the app Terminal (not File Manager), I can see the users Folders
-
I don't excactly get your question. Maybe this screen is the answer?
'datadirectory' => '/media/nextcloudbox',
-
Oh I see the datadirectory was adjusted! I have no clue how this works inside nextcloud if this wasn't done from the first startup of the app as nextcloud maintains database records for each file on disk and only support a single data root. So presumably things got out of sync. You have to reach to nextcloud community for such internals I am afraid, I assumed the volume was used via the external storage plugin: https://docs.cloudron.io/apps/nextcloud/#attaching-external-storage