Cloudron Forum

Any idea? Maybe a bug?

Dec 05 23:12:02 box:backuptask rotateMailBackup: rotating to 2024-12-05-220000-794/mail_v8.1.0
Dec 05 23:12:02 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying with concurrency of 10"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying files from 0-1"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying (multipart) snapshot/mail.tar.gz"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 1 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=0-1073741823"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 2 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=1073741824-2147483647"}
Dec 05 23:12:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 3 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=2147483648-3221225471"}
Dec 05 23:13:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:13:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:13:04 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:14:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 2 - Etag: 77e882104ff405cdbf8d091053169ca7"}
Dec 05 23:14:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 4 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=3221225472-4294967295"}
Dec 05 23:14:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (2) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:14:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (2) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:15:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (1) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:15:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (3) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:15:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (3) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:16:43 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (2) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:17:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (4) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:17:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (4) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:18:03 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (3) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:18:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (5) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:18:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (5) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:19:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (4) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:19:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (6) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:19:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (6) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:20:43 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (5) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:21:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (7) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:21:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (7) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:22:03 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (6) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:22:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (8) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:22:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (8) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:23:23 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (7) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:23:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (9) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:23:45 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (9) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:24:41 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 4 - Etag: a2132197a4d42261637fc197ed066b13"}
Dec 05 23:24:41 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 5 - /cloudronbackup-REMOVEDFORFORUM/snapshot/mail.tar.gz bytes=4294967296-4776642292"}
Dec 05 23:25:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 1 - Etag: f42130621cb6ce886373fc94e97a0940"}
Dec 05 23:25:05 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (10) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:25:09 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copying part 5 - Etag: e60e0408e3be489be7986b759a8f3606"}
Dec 05 23:26:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Retrying (11) multipart copy of snapshot/mail.tar.gz. Error: 504: null 504"}
Dec 05 23:26:25 box:tasks update 2014: {"percent":90.47368421052636,"message":"Aborting multipart copy of snapshot/mail.tar.gz"}
Dec 05 23:26:28 box:storage/s3 copy: s3 copy error when copying snapshot/mail.tar.gz: 504: null
Dec 05 23:26:28 box:tasks update 2014: {"percent":90.47368421052636,"message":"Copied 1 files with error: BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null"}
Dec 05 23:26:28 box:backuptask copy: copied to 2024-12-05-220000-794/mail_v8.1.0 errored. error: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null
Dec 05 23:26:28 box:taskworker Task took 1587.812 seconds
Dec 05 23:26:28 box:tasks setCompleted - 2014: {"result":null,"error":{"stack":"BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null\n at done (/home/yellowtent/box/src/storage/s3.js:333:48)\n at Response.<anonymous> (/home/yellowtent/box/src/storage/s3.js:409:71)\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:367:18)\n at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)\n at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)","name":"BoxError","reason":"External Error","details":{},"message":"Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null"}}
Dec 05 23:26:28 box:tasks update 2014: {"percent":100,"result":null,"error":{"stack":"BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null\n at done (/home/yellowtent/box/src/storage/s3.js:333:48)\n at Response.<anonymous> (/home/yellowtent/box/src/storage/s3.js:409:71)\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:367:18)\n at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)\n at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)","name":"BoxError","reason":"External Error","details":{},"message":"Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null"}}
Dec 05 23:26:28 BoxError: Error copying snapshot/mail.tar.gz (4776642293 bytes): 504 504: null
Dec 05 23:26:28 at done (/home/yellowtent/box/src/storage/s3.js:333:48)
Dec 05 23:26:28 at Response.<anonymous> (/home/yellowtent/box/src/storage/s3.js:409:71)
Dec 05 23:26:28 llowtent/box/node_modules/aws-sdk/lib/request.js:367:18)
Dec 05 23:26:28 at Request.callListeners (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
Dec 05 23:26:28 at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
Dec 05 23:26:28 at Request.emit (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:686:14)
Dec 05 23:26:28 at Request.transition (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:22:10)
Dec 05 23:26:28 at AcceptorStateMachine.runTo (/home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:14:12)
Dec 05 23:26:28 at /home/yellowtent/box/node_modules/aws-sdk/lib/state_machine.js:26:10
Dec 05 23:26:28 at Request.<anonymous> (/home/yellowtent/box/node_modules/aws-sdk/lib/request.js:38:9)

Here is what's going on.

Gotenberg isn't able to start up. I always crashes instantly and give up after a few times. I reinstalled it multiple times now and still no help.

[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - _____ __ __
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - / ___/__ / /____ ___ / / ___ _______ _
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - / (_ / _ \/ __/ -_) _ \/ _ \/ -_) __/ _ '/
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - \___/\___/\__/\__/_//_/_.__/\__/_/ \_, /
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - /___/
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 -
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - A Docker-powered stateless API for PDF files.
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - Version: 8.13.0
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - -------------------------------------------------------
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - [SYSTEM] modules: api chromium exiftool libreoffice libreoffice-api libreoffice-pdfengine logging pdfcpu pdfengines pdftk prometheus qpdf webhook
[no timestamp]  2024-11-08T16:19:51Z mail 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 866 03b9fdda-0a4b-4a05-a704-d2ca51f0d0e0 - [FATAL] provision module api: get routers: provision module chromium: get PDF engine provider: get module: provision module pdfengines: get PDF engines: provision module pdfcpu: PDFCPU_BIN_PATH environment variable is not set
Nov 08 17:19:51 - 2024-11-08 16:19:51,135 INFO exited: gotenberg (exit status 1; not expected)

@luckow said in Add Apache Tika to Paperless-ngx package.:

Testcase: Save eml file to desktop. Open paperless ngx. Upload eml file.

That's how I done it.

@nebulon
I get the following error, when I try to add a eml file

[2024-11-03 17:53:03,119] [ERROR] [paperless.tasks] ConsumeTaskPlugin failed: WG_ aktuelle Zahlen aus dem Fachdienst 35.eml: Error occurred while consuming document WG_ aktuelle Zahlen aus dem Fachdienst 35.eml: Error while converting email to PDF: [Errno 111] Connection refused
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 236, in handle_request
    resp = self._pool.handle_request(req)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 99, in handle_request
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 76, in handle_request
    stream = self._connect(request)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 122, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_backends/sync.py", line 205, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/app/code/src/paperless_mail/parsers.py", line 354, in generate_pdf_from_mail
    .run()
  File "/usr/local/lib/python3.10/dist-packages/gotenberg_client/_base.py", line 113, in run
    resp = self._client.post(url=self._route, headers=self._headers, data=self._form_data, files=self._get_files())
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1157, in post
    return self.request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 837, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 926, in send
    response = self._send_handling_auth(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 954, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 991, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1027, in _send_single_request
    response = transport.handle_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 235, in handle_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/asgiref/sync.py", line 327, in main_wrap
    raise exc_info[1]
  File "/app/code/src/documents/consumer.py", line 476, in run
    document_parser.parse(self.working_copy, mime_type, self.filename)
  File "/app/code/src/paperless_mail/parsers.py", line 183, in parse
    self.archive_path = self.generate_pdf(mail)
  File "/app/code/src/paperless_mail/parsers.py", line 223, in generate_pdf
    mail_pdf_file = self.generate_pdf_from_mail(mail_message)
  File "/app/code/src/paperless_mail/parsers.py", line 357, in generate_pdf_from_mail
    raise ParseError(
documents.parsers.ParseError: Error while converting email to PDF: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/app/code/src/documents/tasks.py", line 148, in consume_file
    msg = plugin.run()
  File "/app/code/src/documents/consumer.py", line 508, in run
    self._fail(
  File "/app/code/src/documents/consumer.py", line 151, in _fail
    raise ConsumerError(f"{self.filename}: {log_message or message}") from exception
documents.consumer.ConsumerError: WG_ aktuelle Zahlen aus dem Fachdienst 35.eml: Error occurred while consuming document WG_ aktuelle Zahlen aus dem Fachdienst 35.eml: Error while converting email to PDF: [Errno 111] Connection refused

@nebulon Is there a possibility to pay 50€ or so to push this request? The possibility to use SOGo on a multi-tenant base is really the only thing keeping my stuck in Plesk.

@nebulon

I looked a bit into it and found the following feature request in the bug tracker, which got implemented.
https://bugs.sogo.nu//view.php?id=3685
https://github.com/Alinto/sogo/commit/d9943e559f42a7d122e9fa30d78579e9159538e4 (Docs commit)

It's adding a placeholder for the domain to the LDAP path. Currently, SOGo access LDAP via "ou=mailboxes,dc=cloudron".
Can you edit Cloudron LDAP so the for example under "ou=example1.com,ou=mailboxes,dc=cloudron" you can find the mailboxes of a specific domain as well? Then you could change the baseDN to "ou=%d,ou=mailboxes,dc=cloudron" and we would have full multi domain support with separation between the domains.

This is great, when you offer SOGo to different companies or different set of people. This way, they cannot see the others.

@nebulon @jdaviescoates SOGo has no access control in Cloudron. It is configured to be used with all users, which owns a mailbox. I can only change dashboard visibility.

To process office and eml files, paperless needs a Apache tika server. IT can be hosted with docker and needs to be added in the paperless config. That's it.

Could you please do that?

https://docs.paperless-ngx.com/configuration/#tika

To clarify, only in the calendar sharing window, where you can add a new user.

Currently, SOGo displays every mail, which is hosted on server to everyone. I would like to hide email from different domains for each other, so that a member of domain1.de doesn't know, that domain2.de and domain3.de even exists. How can I do that?

@girish I got it as a notification in the web interface

Hey, I just got that update notification

I don't really want to install a pre-release version, but I still got that as an update notification. How can I disable getting notifications for pre-release versions and will I get another, when it's not in pre-release anymore?

I asked in the SOGo Mail list. It's hidden. You have to enter you calendar account settings and somewhere, you'll find delegations. There you can enable you shared calendars. Kinda hidden and different for every calendar app.

@nebulon I cannot really post a log, because most of them contains folder names but it seams, that every device goes through all of its folders very often and it gets logged. 10 seconds of log, after the initial sync with the device, contained without spaces, 49534 letters in 53 lines. Mostly LIST and stuff like that.

@nebulon Yes. It works way better. The question still is, how I can block active sync. I have roundabout 30 user on my instance and I don't want one to accidentally ActiveSync.

@nebulon I added three of my accounts on my iPhone. In roundabout 5 seconds, it produced more the 500 lines of log. And this all the time.

@nebulon ActiveSync is kinda hard on resources. How can I disable it, so that my users cannot use it? I tried to edit the nginx.conf in the SOGo pack via terminal, but it's read-only, so I don't wanna mess with it.

Hey,

When I share a calendar to another person, he can see it in the SOGo Webinterface. When I then add the account via CalDAV to my iPhone or any other device, I can only see the calendars of the account itself, not the shared one.

@nebulon Makes sense, just wanted to make sure