Gem installation problem.

Neluser

Hello, there is a problem with installing Gem files in the redmine application.
Plugins just don't find them.
I already had a problem with the Agil plugin, but even after updating the plugin version (with official support, the situation has not changed).
A "clean install" of redmine also didn't help.
To dispel my doubts, I checked the instructions indicated in the documentation and also tried to install only "REDMINE CRM PLUGIN" (https://docs.cloudron.io/apps/redmine/) and as it turned out there also the gem files are not installed correctly and the application goes in reboot.

girish

@neluser mm, I thought I had fixed this one already but clearly not. Let me try to reproduce this and get back.

Neluser

@girish OK thanks.
I was also able to reproduce this problem on my.demo.cloudron.io

jegillikin 0

+1 for reproducing this problem in the wild.

Neluser

@girish Hello, I saw that a new version of Redmine 4.2.2 (2021-08-01) has been released.
Perhaps updating to the latest package will help fix the problem with the gems.

girish

@neluser I am working on the update. Should be out this week.

Neluser

@girish Ok, Thanks!

Neluser

@girish Hello, how is the update going? Is there an improvement in installing gems correctly?

girish

@neluser apologies, I haven't forgotten this. I will work on this tomorrow.

Neluser

@girish Hello again, how are the redmine updates progressing?

girish

@neluser I have pushed out update to 4.2.2. But I don't think the redmine version upgrade will fix the gem installation problem. I made some progress today with trying to figure how to get gems to install out of the plugin directory that redmine can also use. I will continue the work tomorrow.

Neluser

@girish Hello, unfortunately the update did not bring any improvements.
If this information helps, then the latest version on which the gems worked properly is, "Package Version org.redmine.coudronapp@1.6.2 ".

I have suspicions. that the problem might be with the Ruby version.
Because the latest working version of Redmine is installed with the Ruby version: "ruby 2.5.1". And the problems with gems started with the version: "ruby 2.7.0"

girish

@neluser thanks for the hint. Let me see if downgrading ruby will help, but it's a bit complicated because the default ruby on ubuntu 20 is ruby 2.7. Maybe I can try via rvm but I would rather not use older ruby if we can avoid it.

Neluser

This post is deleted!

Neluser

@girish Hello! Are there improvements in the bug fixes?
The Redmine website states that Ruby 2.7 is not supported. Need to use Ruby 2.7.2 or higher.
(Redmine 4.2 does not support Ruby 2.7.0 and 2.7.1. Use Ruby 2.7.2 or higher )
https://www.redmine.org/projects/redmine/wiki/RedmineInstall#fn1

girish

@neluser you are right, the current app docker is not correct and should be updated to use rvm to install a more recent ruby. Looks like ubuntu 20 has ruby 2.7.0 out of the box.

root@6fb7aad07e35:/# ruby2.7 --version
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux-gnu]

I pushed the update to 4.2.3 already, I am updating it to use rvm now.

girish

Looks like https://www.brightbox.com/docs/ruby/ubuntu/#installation is easier

girish

@neluser I have fixed the ruby version issue now. On to the gem installation issue. I get the same error as you:

root@e6152e31-acac-4353-a1c4-8ceab636eed8:/app/code# bundle exec rake redmine:plugins RAILS_ENV=production
Could not find gem 'redmine_crm' in locally installed gems.
Run `bundle install` to install missing gems.
root@e6152e31-acac-4353-a1c4-8ceab636eed8:/app/code# bundle exec rake redmine:plugins NAME=redmine_contacts RAILS_ENV=production
Could not find gem 'redmine_crm' in locally installed gems.
Run `bundle install` to install missing gems.

Neluser

@girish Hi, I also tried to change Ruby version via brightbox to 2.6. But after every reboot of the application, Ruby reverts to version 2.7.0.

girish

@neluser It's not a ruby version issue, it's to do with gem locations. I am still debugging why it can't find the gem anymore, because this used to work before.

Neluser

@girish I noticed that the path has changed in the new version.
Before update

after update

girish

Alright, I am back on this case now. It seems I have re-learn how ruby/gems are deployed each time I investigate this. So, I will leave some notes this time around to help us all later.

girish

OK, I think I figured this one out.

Just leaving my notes here, since I will surely need this few months from now

RubyGems

RubyGems is the package manager and the names of packages is gems. RubyGems is part of ruby since 1.9. The gem CLI lets one install gems. list, info, search install are common subcommands.

By default, gem install requires sudo since into installs into system location /var/lib/gems/2.7.0. It will also install deps of the gem. App can require these gems in code.

Bundler

Bundle(r) allows an app to specify a Gemfile that lists the gems and it's versions. bundle install will install the gems into the same system wide location. bundle exec command runs the executable command in context of the bundle. Meaning, it will ensure that the app does not use any gems not listed in the bundle and also ensure that the gems it is using are compatible with the versions in the Gemfile.

bundler creates Gemfile.lock with "resolved" versions. This has to be writable file even at runtime.

Gem install

gem - GEM_HOME defines where gems are installed. GEM_PATH is list of search locations for gems. This is at the kernel level. Default value of these variables can be got from gem env gemhome and gem env gempath. GEM_HOME is implicitly included into GEM_PATH.
bundler - by default, it's tuned for development and uses gem defaults underneath. This means everything is installing into system locations. bundler has a deployment mode which is a shortcut for installing into vendor/bundle subdirectory (--local will write to <project_root>/.bundle/config and --global will write to ~/.bundle/config). When you call bundle install --deployment first time, it stashes this path info into .bundle/config and all subsequent calls follow this "deployment" mode. In deployment mode, gems will only get loaded from the vendor directory and nowhere else. (As an aside, BUNDLE_PATH env var can be set to tell bundler where to install and locate gems).

Gem loading

Ruby kernel is just loading gems based on LOAD_PATH variable. bundle exec is essentially overloading require calls. It reads Gemfile.lock, patching LOAD_PATH to have paths of the gems and then calling the original kernel require.

girish

OK, next part of notes is how things relate to Redmine package.

Current state:

Redmine is installed into /app/code
gems needed by redmine are installed by calling bundle in deployment mode. This means gems are installed into /app/code/vendor/bundle. The bundle calls creates a .bundle/config to remember deployment mode.
We set BUNDLE_PATH env var to /app/data/vendor/bundle. The intention here is that when the user calls cd plugindir && bundle install, the gems needed by the plugin will get installed into /app/data/vendor/bundle. This works as expected. (We set env var because the .bundle/config won't be found after the cd).
For gem loading to work, GEM_PATH is set to /app/data/vendor/bundle with the intention that bundler exec will use it . But as read in the notes above, the whole point of deployment mode is to only load gems from the path in .bundle/config. This means that gems required by plugins will never be loaded. This is obvious now... we cannot have gems in two locations in deployment mode.

This used to work because this commit switched to deployment mode without understanding the consequences.

The fix:

Install gems needed by redmine into system locations (so developer mode and not deployment mode)
Set GEM_HOME as /app/data/vendor/bundle. This way bundle install inside plugin directory will install there.
Loading also works fine because GEM_HOME is implicitly added into GEM_PATH.

girish

@Neluser When you have the time, can you please try with the latest package? (You can ignore all my notes!) https://docs.cloudron.io/apps/redmine/#installing-plugins is pretty much the same.

girish

In a plot twist worth of O. Henry novel... I went back to using deployment mode. There were two reasons:

Upstream plugin docs like the redmine up plugins recommend running bundle install at the top level.
Using development mode means having to edit a plugin's Gemfile and add source line to make bundle install work.

I fixed the package to now have the bundle in /run/redmine/vendor instead. It's just copied over on startup, it greatly simplifies things.

Neluser

@girish
Hi! The latest version of Redmine is working!
A clean install works as it should!
Updating Redmine from 1.6 to 1.9 also works. The only thing I had to manually delete the "Bundle" folder And after that everything worked as it should.

Thank you for your hard work!

girish

@neluser said in Gem installation problem.:

The only thing I had to manually delete the "Bundle" folder And after that everything worked as it should.

whoops, that is a package bug. I will fix it.

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Solved Gem installation problem.

RubyGems

Bundler

Gem install

Gem loading