Nextcloud App "not responding" after upgrading to Ubuntu 22.04

[ApplegateR]

@andreasb we told that we can't upgrade 22.04 just yet because it wasn't supported.

[girish]

@andreasb Which cloudron version are you on?

(The issue has to do with cgroup v1 / v2 detection)

[jdaviescoates]

@ApplegateR indeed, https://www.cloudron.io/get.html still currently says:

Create a fresh Ubuntu Focal 20.04 x64 server and run these commands

But the linked to guide https://docs.cloudron.io/guides/upgrade-ubuntu-22/ does talk about upgrading. Although looks like it might be a work in progress as it also says e.g.

If you are still on Ubuntu 20, you must first upgrade to Ubuntu 20

Calling @staff to look into this and clarify

[girish]

@jdaviescoates Ubuntu 22 is mostly stable and working but really no rush to update. There is no active issue on Ubuntu 22, I am aware of.

[andreasb]

@girish Cloudron 7.2.4

[andreasb]

@ApplegateR didn't read that anywhere. Let me know if you'd like some information from that machine. If not, I will go back to the snapshot in that case later today.

[girish]

@andreasb thanks! I think the issue is that redis is not starting on ubuntu 22.04. This was already fixed . Can you please check docker images output and check for tag 3.3.0 of redis:

cloudron/redis                       3.3.0                       2187af39ff74   8 weeks ago    2.24GB

Then, docker ps -a should have something like:

2d3003643201   cloudron/redis:3.3.0                                           "/app/code/start.sh"     27 minutes ago   Up 7 minutes                                                                              redis-31ce4921-9b37-41bb-903c-aff587955c00

The 31ce... is the app id of nextcloud. And that redis is using 3.3.0..

[girish]

@andreasb I managed to reproduce this.

To fix this:

# sed -e 's/"version": ".*",/"version":"50.0.0",/' -i /home/yellowtent/platformdata/INFRA_VERSION
# systemctl restart box

The issue is that Ubuntu 22 uses cgroups v2 by default. Containers have to be re-recreated to recognize this change. Changing the infra version triggers a container re-create. Note that it will take a bit for the containers to get re-created. The UI will show "configuring" for all the apps and they will come up one by one.

I have fixed the guide accordingly.

[andreasb]

@girish thx for your support, rebuilding the containers did the job: after restarting, both Redis + NC are up and running again!

[humptydumpty]

@girish When does Cloudron support end for Ubuntu 20.04?

[girish]

@humptydumpty until canonical supports it. That will be for quite a while, see https://ubuntu.com/about/release-cycle

[necrevistonnezr]

@girish said in Nextcloud App "not responding" after upgrading to Ubuntu 22.04:

@andreasb I managed to reproduce this.

To fix this:

# sed -e 's/"version": ".*",/"version":"50.0.0",/' -i /home/yellowtent/platformdata/INFRA_VERSION
# systemctl restart box

The issue is that Ubuntu 22 uses cgroups v2 by default. Containers have to be re-recreated to recognize this change. Changing the infra version triggers a container re-create. Note that it will take a bit for the containers to get re-created. The UI will show "configuring" for all the apps and they will come up one by one.

I have fixed the guide accordingly.

Somehow, this doesn't persist. After a reboot, I see

{
    "version": "49.0.0",
    "baseImages": [
        {
            "repo": "cloudron/base",
            "tag": "cloudron/base:3.2.0@sha256:[REDACTED]"
        }
    ],

in /home/yellowtent/platformdata/INFRA_VERSION

And the mail service does not start.
The error log shows:

Jun 26 18:25:59 cat: /sys/fs/cgroup/memory/memory.limit_in_bytes: No such file or directory

[girish]

@necrevistonnezr It's not supposed to persist. It's just a "hack" to make the code recreate all the containers.

The mail container not starting is indeed a bug when solr/full text search is enabled. You can disable this temporarily.

[necrevistonnezr]

@girish How do I stop Solr in the terminal? I can't stop it in the dashboard - it just shows a spinning circle...

[girish]

@necrevistonnezr can you please try this one liner - https://git.cloudron.io/cloudron/box/-/commit/104bdaf76b1c66b6b7523e3bef0e61f8e18e86a4 and then systemctl restart box.

[necrevistonnezr]

@girish If that replacement should go into /home/yellowtent/platformdata/INFRA_VERSION - no, mail service still not working and showing the same error, even after a reboot.

Jun 27 05:50:14 cat: /sys/fs/cgroup/memory/memory.limit_in_bytes: No such file or directory
Jun 27 05:50:23 cat: /sys/fs/cgroup/memory/memory.limit_in_bytes: No such file or directory
Jun 27 05:50:38 cat: /sys/fs/cgroup/memory/memory.limit_in_bytes: No such file or directory

EDIT: I guess that replacement should go into /home/yellowtent/box/src/infra_version.js - but after a restart of box, still not running...

EDIT 2: After 30 minutes or so, mail service is running again!

[girish]

@necrevistonnezr said in Nextcloud App "not responding" after upgrading to Ubuntu 22.04:

EDIT: I guess that replacement should go into /home/yellowtent/box/src/infra_version.js - but after a restart of box, still not running...

Yes, that's the correct place to fix it!

I will add a warning to our docs that updating to ubuntu 22 when using FTS feature does not work out of the box / without patching until 7.3.