Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Docker service failed after docker-compose install

    Support
    docker
    4
    6
    1435
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Z
      ZeZaung last edited by girish

      I was installing docker-compose on my Cloudron installation and managed to break everything. Thinking that uninstalling docker-compose would fix it, I gave it a reboot and now I have multiple services failing to start.

      I cannot start unbound.

      root@cloudron0:~# docker -v
      Docker version 20.10.3, build 48d30b5

      root@cloudron0:/var/lib/docker# systemctl restart docker
      Job for docker.service failed because the control process exited with error code.
      See "systemctl status docker.service" and "journalctl -xe" for details.
      root@cloudron0:/var/lib/docker# systemctl status docker.service
      ● docker.service - LSB: Create lightweight, portable, self-sufficient containers.
         Loaded: loaded (/etc/init.d/docker; generated)
        Drop-In: /etc/systemd/system/docker.service.d
                 └─cloudron.conf
         Active: failed (Result: exit-code) since Fri 2021-08-06 14:44:41 UTC; 9s ago
           Docs: man:systemd-sysv-generator(8)
        Process: 19133 ExecStart=/usr/bin/dockerd -H fd:// --log-driver=journald --exec-opt native.cgroupdriver=cgroupfs --sto
      
      Aug 06 14:44:41 cloudron0 systemd[1]: Starting LSB: Create lightweight, portable, self-sufficient containers....
      Aug 06 14:44:41 cloudron0 systemd[19133]: docker.service: Failed to execute command: No such file or directory
      Aug 06 14:44:41 cloudron0 systemd[19133]: docker.service: Failed at step EXEC spawning /usr/bin/dockerd: No such file or
      Aug 06 14:44:41 cloudron0 systemd[1]: docker.service: Control process exited, code=exited status=203
      Aug 06 14:44:41 cloudron0 systemd[1]: docker.service: Failed with result 'exit-code'.
      Aug 06 14:44:41 cloudron0 systemd[1]: Failed to start LSB: Create lightweight, portable, self-sufficient containers..
      
      root@cloudron0:/var/lib/docker# systemctl status box
      ● box.service - Cloudron Admin
         Loaded: loaded (/etc/systemd/system/box.service; enabled; vendor preset: enabled)
         Active: active (running) since Fri 2021-08-06 14:43:31 UTC; 745ms ago
       Main PID: 16120 (node)
          Tasks: 7 (limit: 4915)
         CGroup: /system.slice/box.service
                 └─16120 node /home/yellowtent/box/box.js
      
      Aug 06 14:43:31 cloudron0 systemd[1]: Started Cloudron Admin.
      root@cloudron0:/var/lib/docker# systemctl status unbound
      ● unbound.service - Unbound DNS Resolver
         Loaded: loaded (/etc/systemd/system/unbound.service; enabled; vendor preset: enabled)
         Active: failed (Result: exit-code) since Fri 2021-08-06 14:31:38 UTC; 12min ago
        Process: 23072 ExecStart=/usr/sbin/unbound -d (code=exited, status=1/FAILURE)
       Main PID: 23072 (code=exited, status=1/FAILURE)
      
      Aug 06 14:31:37 cloudron0 systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
      Aug 06 14:31:37 cloudron0 systemd[1]: unbound.service: Failed with result 'exit-code'.
      Aug 06 14:31:38 cloudron0 systemd[1]: unbound.service: Service hold-off time over, scheduling restart.
      Aug 06 14:31:38 cloudron0 systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
      Aug 06 14:31:38 cloudron0 systemd[1]: Stopped Unbound DNS Resolver.
      Aug 06 14:31:38 cloudron0 systemd[1]: unbound.service: Start request repeated too quickly.
      Aug 06 14:31:38 cloudron0 systemd[1]: unbound.service: Failed with result 'exit-code'.
      Aug 06 14:31:38 cloudron0 systemd[1]: Failed to start Unbound DNS Resolver.
      root@cloudron0:/var/lib/docker# unbound-anchor -a /var/lib/unbound/root.key
      root@cloudron0:/var/lib/docker# systemctl restart unbound
      root@cloudron0:/var/lib/docker# systemctl status unbound
      ● unbound.service - Unbound DNS Resolver
         Loaded: loaded (/etc/systemd/system/unbound.service; enabled; vendor preset: enabled)
         Active: failed (Result: exit-code) since Fri 2021-08-06 14:43:53 UTC; 2s ago
        Process: 17316 ExecStart=/usr/sbin/unbound -d (code=exited, status=1/FAILURE)
       Main PID: 17316 (code=exited, status=1/FAILURE)
      
      Aug 06 14:43:52 cloudron0 systemd[1]: unbound.service: Main process exited, code=exited, status=1/FAILURE
      Aug 06 14:43:52 cloudron0 systemd[1]: unbound.service: Failed with result 'exit-code'.
      Aug 06 14:43:53 cloudron0 systemd[1]: unbound.service: Service hold-off time over, scheduling restart.
      Aug 06 14:43:53 cloudron0 systemd[1]: unbound.service: Scheduled restart job, restart counter is at 5.
      Aug 06 14:43:53 cloudron0 systemd[1]: Stopped Unbound DNS Resolver.
      Aug 06 14:43:53 cloudron0 systemd[1]: unbound.service: Start request repeated too quickly.
      Aug 06 14:43:53 cloudron0 systemd[1]: unbound.service: Failed with result 'exit-code'.
      Aug 06 14:43:53 cloudron0 systemd[1]: Failed to start Unbound DNS Resolver.
      

      MySQL and Nginx are running, and the files in /home/yellowtent/appsdata are intact

      Any help would be appreciated.

      girish 1 Reply Last reply Reply Quote 0
      • girish
        girish Staff @ZeZaung last edited by

        @zezaung Looks like maybe uninstalling compose also nuked docker from what I can tell. The logs say docker.service: Failed at step EXEC spawning /usr/bin/dockerd . I guess that means the docker binary disappeared. Can you check if /var/lib/docker is still there? If that too is gone, we have to re-create all the containers (I can explain how to do this later).

        We install docker from the upstream repos directory, so download the 3 debs below and install them using apt install -y . docker_version below is 20.10.3 and ubuntu_codename is the output of lsb_release -cs

        https://download.docker.com/linux/ubuntu/dists/${ubuntu_codename}/pool/stable/amd64/containerd.io_1.4.3-1_amd64.deb
        https://download.docker.com/linux/ubuntu/dists/${ubuntu_codename}/pool/stable/amd64/docker-ce-cli_${docker_version}~3-0~ubuntu-${ubuntu_codename}_amd64.deb
        https://download.docker.com/linux/ubuntu/dists/${ubuntu_codename}/pool/stable/amd64/docker-ce_${docker_version}~3-0~ubuntu-${ubuntu_codename}_amd64.deb
        
        1 Reply Last reply Reply Quote 0
        • girish
          girish Staff last edited by

          If it helps, the relevant code we use is here - https://git.cloudron.io/cloudron/box/-/blob/master/scripts/installer.sh#L77 (but you cannot run the installer script directly since it's meant to be called from a new box update).

          Z 1 Reply Last reply Reply Quote 0
          • Z
            ZeZaung @girish last edited by

            @girish I can't thank you enough. Had some difficulties with the -o /tmp/filename.deb (it wasn't saving the file) so I downloaded and renamed the .deb onto my computer, filezilla'd to my home directory, then used mv filename.deb /tmp

            I restarted unbound and everything started running perfectly

            Here is the result:
            The lib files seemed untouched

            root@cloudron0:/var/lib/docker# ls -al
            total 200
            drwx--x--x  14 root root  4096 Aug  6 14:31  .
            drwxr-xr-x  47 root root  4096 Dec 10  2020  ..
            drwx------   2 root root  4096 Dec 10  2020  builder
            drwx--x--x   4 root root  4096 Mar 15 03:35  buildkit
            drwx-----x  59 root root 12288 Aug  6 13:32  containers
            drwx------   3 root root  4096 Dec 10  2020  image
            drwxr-x---   3 root root  4096 Dec 10  2020  network
            drwx-----x 507 root root 73728 Aug  6 13:57  overlay2
            drwx------   4 root root  4096 Dec 10  2020  plugins
            drwx------   2 root root  4096 Aug  6 13:57  runtimes
            drwx------   2 root root  4096 Dec 10  2020  swarm
            drwx------   2 root root  4096 Aug  6 13:57  tmp
            drwx------   2 root root  4096 Dec 10  2020  trust
            drwx-----x 458 root root 61440 Aug  6 13:57  volumes
            -rw-r--r--   1 root root  1394 Aug  6 14:31 'ystemctl status unbound'
            

            I hope this worked the way it should:

            root@cloudron0:/tmp# apt install -y /tmp/containerd.deb
            Reading package lists... Done
            Building dependency tree
            Reading state information... Done
            Note, selecting 'containerd.io' instead of '/tmp/containerd.deb'
            The following packages were automatically installed and are no longer required:
              amd64-microcode aufs-tools cgroupfs-mount golang-docker-credential-helpers intel-microcode iucode-tool libsecret-1-0
              libsecret-common pigz python-asn1crypto python-backports.ssl-match-hostname python-cached-property python-certifi
              python-cffi-backend python-chardet python-cryptography python-docker python-dockerpty python-dockerpycreds
              python-docopt python-enum34 python-funcsigs python-functools32 python-idna python-ipaddress python-jsonschema
              python-mock python-openssl python-pbr python-pkg-resources python-requests python-six python-texttable
              python-urllib3 python-websocket python-yaml
            Use 'apt autoremove' to remove them.
            The following NEW packages will be installed:
              containerd.io
            0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded.
            After this operation, 129 MB of additional disk space will be used.
            Get:1 /tmp/containerd.deb containerd.io amd64 1.4.3-1 [28.1 MB]
            Selecting previously unselected package containerd.io.
            (Reading database ... 151160 files and directories currently installed.)
            Preparing to unpack /tmp/containerd.deb ...
            Reading package lists... Done
            Building dependency tree
            Reading state information... Done
            Note, selecting 'docker-ce-cli' instead of '/tmp/docker-ce-cli.deb'
            docker-ce-cli is already the newest version (5:20.10.3~3-0~ubuntu-bionic).
            The following packages were automatically installed and are no longer required:
              amd64-microcode aufs-tools cgroupfs-mount golang-docker-credential-helpers intel-microcode iucode-tool libsecret-1-0
              libsecret-common pigz python-asn1crypto python-backports.ssl-match-hostname python-cached-property python-certifi
              python-cffi-backend python-chardet python-cryptography python-docker python-dockerpty python-dockerpycreds
              python-docopt python-enum34 python-funcsigs python-functools32 python-idna python-ipaddress python-jsonschema
              python-mock python-openssl python-pbr python-pkg-resources python-requests python-six python-texttable
              python-urllib3 python-websocket python-yaml
            Use 'apt autoremove' to remove them.
            0 upgraded, 0 newly installed, 0 to remove and 43 not upgraded.
            root@cloudron0:/tmp# apt install -y /tmp/docker.deb
            Reading package lists... Done
            Building dependency tree
            Reading state information... Done
            Note, selecting 'docker-ce' instead of '/tmp/docker.deb'
            The following packages were automatically installed and are no longer required:
              amd64-microcode golang-docker-credential-helpers intel-microcode iucode-tool libsecret-1-0 libsecret-common
              python-asn1crypto python-backports.ssl-match-hostname python-cached-property python-certifi python-cffi-backend
              python-chardet python-cryptography python-docker python-dockerpty python-dockerpycreds python-docopt python-enum34
              python-funcsigs python-functools32 python-idna python-ipaddress python-jsonschema python-mock python-openssl
              python-pbr python-pkg-resources python-requests python-six python-texttable python-urllib3 python-websocket
              python-yaml
            Use 'apt autoremove' to remove them.
            Recommended packages:
              docker-ce-rootless-extras
            The following NEW packages will be installed:
              docker-ce
            0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded.
            After this operation, 121 MB of additional disk space will be used.
            Get:1 /tmp/docker.deb docker-ce amd64 5:20.10.3~3-0~ubuntu-bionic [24.8 MB]
            Selecting previously unselected package docker-ce.
            (Reading database ... 151174 files and directories currently installed.)
            Preparing to unpack /tmp/docker.deb ...
            Unpacking docker-ce (5:20.10.3~3-0~ubuntu-bionic) ...
            Setting up docker-ce (5:20.10.3~3-0~ubuntu-bionic) ...
            Processing triggers for systemd (237-3ubuntu10.43) ...#############################.................................]
            Processing triggers for ureadahead (0.100.0-21) ...
            
            root@cloudron0:/tmp# systemctl status box
            ● box.service - Cloudron Admin
               Loaded: loaded (/etc/systemd/system/box.service; enabled; vendor preset: enabled)
               Active: active (running) since Fri 2021-08-06 18:49:09 UTC; 3min 8s ago
             Main PID: 21658 (node)
                Tasks: 11 (limit: 4915)
               CGroup: /system.slice/box.service
                       └─21658 node /home/yellowtent/box/box.js
            
            Aug 06 18:49:09 cloudron0 systemd[1]: Started Cloudron Admin.
            Aug 06 18:49:16 cloudron0 sudo[24775]: pam_unix(sudo:session): session opened for user root by (uid=0)
            Aug 06 18:49:16 cloudron0 sudo[24775]: pam_unix(sudo:session): session closed for user root
            Aug 06 18:49:16 cloudron0 sudo[24838]: pam_unix(sudo:session): session opened for user root by (uid=0)
            Aug 06 18:49:16 cloudron0 sudo[24838]: pam_unix(sudo:session): session closed for user root
            Aug 06 18:49:47 cloudron0 sudo[29454]: pam_unix(sudo:session): session opened for user root by (uid=0)
            Aug 06 18:49:47 cloudron0 sudo[29454]: pam_unix(sudo:session): session closed for user root
            root@cloudron0:/tmp# systemctl status unbound
            ● unbound.service - Unbound DNS Resolver
               Loaded: loaded (/etc/systemd/system/unbound.service; enabled; vendor preset: enabled)
               Active: failed (Result: exit-code) since Fri 2021-08-06 14:45:47 UTC; 4h 7min ago
             Main PID: 1346 (code=exited, status=1/FAILURE)
            
            Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
            root@cloudron0:/tmp# unbound-anchor -a /var/lib/unbound/root.key
            boundroot@cloudron0:/tmp# systemctl restart unbound
            root@cloudron0:/tmp# systemctl status unbound
            ● unbound.service - Unbound DNS Resolver
               Loaded: loaded (/etc/systemd/system/unbound.service; enabled; vendor preset: enabled)
               Active: active (running) since Fri 2021-08-06 18:53:01 UTC; 3s ago
             Main PID: 4506 (unbound)
                Tasks: 1 (limit: 4915)
               CGroup: /system.slice/unbound.service
                       └─4506 /usr/sbin/unbound -d
            
            Aug 06 18:53:01 cloudron0 systemd[1]: Started Unbound DNS Resolver.
            Aug 06 18:53:01 cloudron0 unbound[4506]: [4506:0] notice: init module 0: subnet
            Aug 06 18:53:01 cloudron0 unbound[4506]: [4506:0] notice: init module 1: validator
            Aug 06 18:53:01 cloudron0 unbound[4506]: [4506:0] notice: init module 2: iterator
            Aug 06 18:53:01 cloudron0 unbound[4506]: [4506:0] info: start of service (unbound 1.6.7).
            root@cloudron0:/tmp# systemctl status nginx
            ● nginx.service - nginx - high performance web server
               Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
              Drop-In: /etc/systemd/system/nginx.service.d
                       └─cloudron.conf
               Active: active (running) since Fri 2021-08-06 14:45:45 UTC; 4h 7min ago
                 Docs: http://nginx.org/en/docs/
             Main PID: 980 (nginx)
                Tasks: 5 (limit: 4915)
               CGroup: /system.slice/nginx.service
                       ├─  980 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
                       ├─29499 nginx: worker process
                       ├─29500 nginx: worker process
                       ├─29501 nginx: worker process
                       └─29502 nginx: worker process
            
            Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
            root@cloudron0:/tmp# systemctl status mysql
            ● mysql.service - MySQL Community Server
               Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
               Active: active (running) since Fri 2021-08-06 14:45:44 UTC; 4h 7min ago
             Main PID: 756 (mysqld)
                Tasks: 32 (limit: 4915)
               CGroup: /system.slice/mysql.service
                       └─756 /usr/sbin/mysqld --daemonize --pid-file=/run/mysqld/mysqld.pid
            
            Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
            
            1 Reply Last reply Reply Quote 1
            • J
              johannesjom last edited by johannesjom

              Today I also managed to completely destroy Cloudron by installing Docker-Compose (apt install docker-compose) today. I think you should prevent exactly that during the installation.

              nebulon 1 Reply Last reply Reply Quote 0
              • nebulon
                nebulon Staff @johannesjom last edited by

                @johannesjom generally it is not supported to install other things via apt manually. There are some warnings about this when you SSH into your server, but we can't quite prevent users who have root access to their own server to install things or also change essential config files. So not sure what we can do here besides warning the user.

                1 Reply Last reply Reply Quote 1
                • First post
                  Last post
                Powered by NodeBB