Gitlab - How to restore / migrate a non Cloudron Gitlab

[girish]

@brutalbirdie If you open that database.sql file, there will be a LOT of userxxxx references (referring to the old database user). You can to replace all with the new database username.

[girish]

@brutalbirdie So, in the database.sql.gz, there are a lots of lines like ALTER FUNCTION public.trigger_21e7a2602957() OWNER TO user57c829faa1694c05980bbc6113b30ae2; (and many other OWNER stuff). The user57c829faa1694c05980bbc6113b30ae2 is the postgres username of the old installation. It has to be changed to the new postgres username from Cloudron. Mine was user970fb4e4a4654d1087af2cb720a159e1

Something like: sed -e 's/user57c829faa1694c05980bbc6113b30ae2/user970fb4e4a4654d1087af2cb720a159e1/' -i database.sql.

[BrutalBirdie]

@girish Problem is the DB I try to fix had the user gitlab and replacing just the string gitlab would destroy the DB.
If only the OWNER TO $DB_USER needs to be changed, then all good.

But from your answer I am still not sure if this is the only part where the user needs to be changed.

[girish]

@brutalbirdie ah, I understand your predicament now. That's correct only the OWNER TO needs to be replaced.

[BrutalBirdie]

@girish TOP! Will try it out on Monday.

[BrutalBirdie]

@brutalbirdie

2021-09-27 11:16:52 +0000 -- Cleaning the database ... 
2021-09-27 11:16:54 +0000 -- done
2021-09-27 11:16:54 +0000 -- Restoring database ... 
Restoring PostgreSQL database dbd5499e3cf81b43d093724d69fa223688 ... ERROR:  must be owner of extension btree_gist
ERROR:  must be owner of extension pg_trgm
. . .
ALTER TABLE
ALTER TABLE
[DONE]
2021-09-27 11:17:24 +0000 -- done
2021-09-27 11:17:24 +0000 -- Restoring repositories ...
rake aborted!
TypeError: no implicit conversion of nil into String
/home/git/gitlab/lib/backup/gitaly_backup.rb:75:in `absolute_path'
/home/git/gitlab/lib/backup/gitaly_backup.rb:75:in `bin_path'
/home/git/gitlab/lib/backup/gitaly_backup.rb:28:in `start'
/home/git/gitlab/lib/backup/repositories.rb:47:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:124:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:80:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
Tasks: TOP => gitlab:backup:repo:restore

hmm

[BrutalBirdie]

I added a full --trace log here https://paste.cloudron.dev/mocukugoho.md

[girish]

@brutalbirdie great success! that exact error is what I fixed in package v1.55.3 . Are you using that package version?

I think you can also make it work in older packages with adding the line in /app/data/gitlab.yml:

backup:
    gitaly_backup_path: /home/git/gitaly/gitaly-backup

[BrutalBirdie]

@girish

App Title and Version
GitLab 14.2.3
App ID
d5499e3c-f81b-43d0-9372-4d69fa223688
Package Version
com.gitlab.cloudronapp@1.55.3
Last Updated
3 days ago

Btw. The Gitlab Version in this information is wrong, the version should be 14.2.4 since this is the version running in this release tag.
So I am running the v1.55.3
I will add gitaly_backup_path: /home/git/gitaly/gitaly-backup to backup: and try again, after a app restart.

[BrutalBirdie]

@girish
The last error

2021-09-28 06:36:02 +0000 -- Restoring container registry images ... 
rake aborted!
Errno::ENOENT: No such file or directory - /app/data/backups/registry.tar.gz
/home/git/gitlab/lib/backup/files.rb:95:in `run_pipeline!'
/home/git/gitlab/lib/backup/files.rb:60:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:277:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:86:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
Tasks: TOP => gitlab:backup:registry:restore
(See full trace by running task with --trace)

But I think I can ignore this one since the old gitlab simply did not backup the registry.
Now the gitlab goes thru testing by his Owner.

[BrutalBirdie]

@girish The https://DOMAIN.TLD//admin/runners return error code 500.
This should be a problem with the migrated secrets.
The secrets where a gitlab-secrets.json I converted the json to yaml.

After that I noticed that gitlab_rails should be production.
Some more testing...

[BrutalBirdie]

Ahhh finally.
I deleted the registry: part in the secrets.yaml and now runners are accessible again!

After a final testing by the owner, I will write up a doc for this type of migration.

[BrutalBirdie]

Looks like we are all good now.
I will submit a PR for the docs on gitlab for a migration step.

[BrutalBirdie]

@girish

https://git.cloudron.io/cloudron/docs/-/merge_requests/28

I created a PR for this topic as a guide.
Have fun reading and fixing my mistakes (if there is something to change hit me up in a dm/chat/PR)

[girish]

@brutalbirdie thanks! I have merged it but I will move it to a separate Guide doc.