Mastodon 4.2.0 update error

[tranquilbigfoot 0]

Hi everyone

I had an auto update from v1.11.11 (4.1.9) to v1.12.0 (4.2.0) last night and the app is stuck on starting.

I downgraded to 4.1.9 to resolve the issue and manually updated to 4.2.0, but I'm getting the same error.

From the logs, I'm getting this on a loop:

Sep 23 10:07:02==> Configuring mastodon
Sep 23 10:07:02==> Migrating database
Sep 23 10:07:04warning: parser/current is loading parser/ruby30, which recognizes 3.0.6-compliant syntax, but you are running 3.0.4.
Sep 23 10:07:04Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Sep 23 10:07:05I, [2023-09-23T08:07:05.652221 #13] INFO -- : Migrating to AddPrimaryKeyToStatusesTagsJoinTable (20230531154811)
Sep 23 10:07:05== 20230531154811 AddPrimaryKeyToStatusesTagsJoinTable: migrating =============
Sep 23 10:07:05[strong_migrations] DANGER: No lock timeout set
Sep 23 10:07:05-- execute("ALTER TABLE statuses_tags ADD PRIMARY KEY USING INDEX index_statuses_tags_on_tag_id_and_status_id")
Sep 23 10:07:05rails aborted!
Sep 23 10:07:05StandardError: An error has occurred, all later migrations canceled:
Sep 23 10:07:05
Sep 23 10:07:05PG::UndefinedObject: ERROR: index "index_statuses_tags_on_tag_id_and_status_id" does not exist
Sep 23 10:07:05LINE 1: ALTER TABLE statuses_tags ADD PRIMARY KEY USING INDEX index_...
Sep 23 10:07:05^
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:9:in `block (2 levels) in up'
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:8:in `block in up'
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:7:in `up'
Sep 23 10:07:05
Sep 23 10:07:05Caused by:
Sep 23 10:07:05ActiveRecord::StatementInvalid: PG::UndefinedObject: ERROR: index "index_statuses_tags_on_tag_id_and_status_id" does not exist
Sep 23 10:07:05LINE 1: ALTER TABLE statuses_tags ADD PRIMARY KEY USING INDEX index_...
Sep 23 10:07:05^
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:9:in `block (2 levels) in up'
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:8:in `block in up'
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:7:in `up'
Sep 23 10:07:05
Sep 23 10:07:05Caused by:
Sep 23 10:07:05PG::UndefinedObject: ERROR: index "index_statuses_tags_on_tag_id_and_status_id" does not exist
Sep 23 10:07:05LINE 1: ALTER TABLE statuses_tags ADD PRIMARY KEY USING INDEX index_...
Sep 23 10:07:05^
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:9:in `block (2 levels) in up'
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:8:in `block in up'
Sep 23 10:07:05/app/code/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb:7:in `up'
Sep 23 10:07:05Tasks: TOP => db:migrate
Sep 23 10:07:05(See full trace by running task with --trace)
Sep 23 10:07:17=> Healtheck error: Error: Timeout of 7000ms exceeded
Sep 23 10:07:27=> Healtheck error: Error: Timeout of 7000ms exceeded
Sep 23 10:07:37=> Healtheck error: Error: Timeout of 7000ms exceeded
Sep 23 10:07:43=> Healtheck error: Error: connect EHOSTUNREACH 172.18.17.237:8000
Sep 23 10:07:52=> Healtheck error: Error: connect EHOSTUNREACH 172.18.17.237:8000
Sep 23 10:08:01=> Healtheck error: Error: connect EHOSTUNREACH 172.18.17.237:8000

Any ideas on how I can fix it myself, or if I should wait on 4.2.x later on?

Thanks!

[nebulon]

hm manual and automatic updates are performing the same steps in the background, so that is to be expected that they behave the same way. Looks like then the issue is at least reproducible on your instance.

The migration file in question is https://github.com/mastodon/mastodon/blob/4b7bc1f07c712cf2375882da4f12a8af29444590/db/migrate/20230531154811_add_primary_key_to_statuses_tags_join_table.rb

There doesn't seem to be an upstream issue so far and the index index_statuses_tags_on_tag_id_and_status_id is nowhere to be found otherwise. How old is your mastodon installation? Looks like we have to report this upstream.

Does anyone else here experience the same issue, as at least on the instances I am actively using, this didn't happen.

[jdaviescoates]

Quite possibly just need to wait, see https://forum.cloudron.io/post/74060

[nebulon]

This unfortunately will not apply here, since the db migration is an atomic step required to even start the app. For some reason that index does not exist on that instance.

[tranquilbigfoot 0]

@nebulon Thanks! It's been running on Cloudron since 1.4.2 (Mastodon 3.2.1) and it's been auto updating like clockwork since then. It was a fresh install back in the day with no migrations. I'll do some more investigation about the DB issue (I'll use the official upgrade FAQ now) and report back if I find a fix.

[tranquilbigfoot 0]

Found a solution!

Went into my 4.1.9 app page in Cloudron, clicked Terminal, used the Postgres button to get into DB mode, and typed this in:

CREATE UNIQUE INDEX index_statuses_tags_on_tag_id_and_status_id ON public.statuses_tags USING btree (tag_id, status_id);

Managed to upgrade seamlessly to 4.2.0 after that.