Email configured, but no notification

[nebulon]

Not an OpenProject expert, but maybe there is a setting for invites or such? If the test email works I suppose the package setup is correct. Otherwise you may also reach out to OpenProject upstream to clarify what is expected to happen.

[umnz]

@nebulon Good call. I checked upstream to see if this was a common issue (I had assumed it was Cloudron related) - and here's what I found.

Docker / Worker / Port Issue
https://community.openproject.org/topics/11644

Using this test URL you can run a health check on an OpenProject instance:
https://openproject.example.com/health_checks/all

full
  database: PASSED Schema version: 20240611105232 (0.005s)
  default: PASSED Application is running (0.000s)
  mail: PASSED ActionMailer::Base check to mail:2525 successful (0.004s)
  puma: PASSED N/A as Puma is not used. (0.000s) (OPTIONAL)
  worker: FAILED No good_job processes are active. (0.048s)
  worker_backed_up: FAILED 199 jobs are waiting to be picked up for more than 5 minutes. (0.024s) (OPTIONAL)

This issue is caused by the worker not working, and this is happening on fresh deployments of OpenProject installed on Ubuntu 24.04 LTS.

Can you please check on a fresh install of Cloudron at your end to see whether or not your worker is running?

Sending a test email will work - as it is called directly and not via the worker. But creating & inviting a test user (plus many other operations) will add to the worker's queue which are failing.

Also, as others have reported, there is a recurring issue in the Cloudron logs for the worker:

Jul 17 14:35:13 => Run cron job
Jul 17 14:35:27 rake aborted!
Jul 17 14:35:27 Don't know how to build task 'jobs:workoff' (See the list of available tasks with `rake --tasks`)
Jul 17 14:35:27 (See full trace by running task with --trace)

Note the above Cron is Cloudron's call - not one I have set up. A fresh installation will get this error and I believe it is the source of the emails not sending out when queued.

[umnz]

Further reading about the cronjob here:
https://community.openproject.org/topics/5276

root@xxx:/app/code# bundle exec rake jobs:workoff
rake aborted!
Don't know how to build task 'jobs:workoff' (See the list of available tasks with `rake --tasks`)
/app/code/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/exe/rake:27:in `<top (required)>'
/root/.rbenv/versions/3.3.2/bin/bundle:25:in `load'
/root/.rbenv/versions/3.3.2/bin/bundle:25:in `<main>'
(See full trace by running task with --trace)

[umnz]

Assuming it was a cron related issue, I found this post:
https://community.openproject.org/topics/5276?r=5306#message-5306

Where it was recommended to run the following commands - to ensure various services were running:

RAILS_ENV="production" bundle exec rake db:create:all
RAILS_ENV="production" bundle exec rake db:migrate
RAILS_ENV="production" bundle exec rake db:seed
RAILS_ENV="production" bundle exec rake assets:precompile

All but the last command is successful. I get an error which aborts the process:

root@xxx:/app/code# RAILS_ENV="production" bundle exec rake assets:precompile
I, [2024-07-17T03:15:54.881512 #446]  INFO -- : Increasing database pool size to 17 to match max threads
Exporting I18n.js locales
=> Done in 12.29s
Linking frontend plugins
Cleaning linked target directory /app/code/frontend/src/app/features/plugins/linked
Linking frontend of OpenProject plugin budgets to /app/code/frontend/src/app/features/plugins/linked/budgets.
rake aborted!
Errno::EROFS: Read-only file system @ rb_file_s_symlink - (/app/code/modules/budgets/frontend/module, /app/code/frontend/src/app/features/plugins/linked/budgets/module) (Errno::EROFS)
/app/code/lib/open_project/plugins/frontend_linking/generator.rb:74:in `block (2 levels) in regenerate_angular_links'
/app/code/lib/open_project/plugins/frontend_linking/generator.rb:69:in `each'
/app/code/lib/open_project/plugins/frontend_linking/generator.rb:69:in `block in regenerate_angular_links'
/app/code/lib/open_project/plugins/frontend_linking/generator.rb:61:in `regenerate_angular_links'
/app/code/lib/open_project/plugins/frontend_linking/generator.rb:52:in `regenerate!'
/app/code/lib/open_project/plugins/frontend_linking.rb:35:in `regenerate!'
/app/code/lib/tasks/plugins.rake:35:in `block (3 levels) in <top (required)>'
/app/code/lib/tasks/assets.rake:61:in `block (2 levels) in <top (required)>'
/app/code/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/exe/rake:27:in `<top (required)>'
/root/.rbenv/versions/3.3.2/bin/bundle:25:in `load'
/root/.rbenv/versions/3.3.2/bin/bundle:25:in `<main>'
Tasks: TOP => openproject:plugins:register_frontend
(See full trace by running task with --trace)

I'm not 100% sure what happens during the deployment of OpenProject via Cloudron so I can only leave this with you guys to help me understand how to get the cron working correctly.

[nebulon]

Thanks for all the research! Most rake commands are run already in Cloudron, however the job workers are indeed not run.

The command would be RAILS_ENV="production" bundle exec good_job start which is a longstanding task, so we need to add support for that in the package, which currently is working with one apache+passenger and a cron script.

We will also start using the better healtheck path you mentioned.

[umnz]

Thanks @nebulon
Using that command via terminal allowed me to clear out the backlog of tasks -- but it didn't help solve system email mystery. I left that worker running while attempting to send an invite and also a password reset and still no emails are leaving the system. Of course, directly calling a test email works. So I'm kind of at a loss at this point.

[umnz]

I've found some additional information here that might be of help @nebulon but I'm not sure how to run the openproject commands they mention.

https://www.openproject.org/docs/installation-and-operations/installation-faq/#i-dont-receive-emails-test-email-works-fine-but-not-the-one-for-work-package-updates-what-can-i-do

[nebulon]

I am currently reworking the package to be closer to upstream. Using puma instead of passenger for example.
For the imap fetching in cron, I will see how we can integrate https://github.com/opf/openproject/blob/3757cf397f0b7d048aa7b180cfb100c06103eb3c/docker/prod/cron

[nebulon]

The email sending via the task worker should be fixed now with latest package release.

[umnz]

Hi @nebulon - thank you! Just confirming after updating, the emails are now being sent.