Moving docker containers crashes apps



  • I used this instructions on cloudron docs. Apps not working after starting docker or rebooting machine.docker ps has no container running.

    Moving docker images to another location
    Cloudron uses Docker for containerizing applications and docker images tend to consume a lot >of space. The docker images are stored by default at /var/lib/docker. They can be moved to an >external storage as follows:

    Stop docker
    systemctl stop docker

    Move existing contents to external storage. Any external storage must be formatted as ext4. >Assuming, /mnt/docker is the new location:
    rsync -aHSX /var/lib/docker /mnt/docker

    Link /var/lib/docker to the new location
    rm -rf /var/lib/docker
    ln -s /mnt/docker /var/lib/docker

    Alternately, you can bind mount mount --bind /mnt/docker /var/lib/docker. This, however, >requires adding an entry in /etc/fstab to persist across reboots.

    Start docker
    systemctl start docker

    I get this error on docker log :

    rd process" pid=1564
    489c9c1b21a6d78b5c538cd395416ec50f88 version=v1.0.3
    " module=containerd type=io.containerd.content.v1
    .." module=containerd type=io.containerd.snapshotter.v1
    1.btrfs" error="path /mnt/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module=containerd
    fs"..." module=containerd type=io.containerd.snapshotter.v1
    module=containerd type=io.containerd.metadata.v1
    n" error="path /mnt/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module="containerd/io.containerd.metadata.v1.bolt"
     module=containerd type=io.containerd.differ.v1
    odule=containerd type=io.containerd.gc.v1
    " module=containerd type=io.containerd.grpc.v1
    odule=containerd type=io.containerd.grpc.v1
    le=containerd type=io.containerd.grpc.v1
    dule=containerd type=io.containerd.grpc.v1
    ." module=containerd type=io.containerd.grpc.v1
    dule=containerd type=io.containerd.grpc.v1
    dule=containerd type=io.containerd.grpc.v1
    " module=containerd type=io.containerd.grpc.v1
     module=containerd type=io.containerd.grpc.v1
    " module=containerd type=io.containerd.monitor.v1
    module=containerd type=io.containerd.runtime.v1
    ule=containerd type=io.containerd.grpc.v1
    odule=containerd type=io.containerd.grpc.v1
    ..." module=containerd type=io.containerd.grpc.v1
    -containerd-debug.sock" module="containerd/debug"
    -containerd.sock" module="containerd/grpc"
    containerd
    took 0.00 seconds"
    period"
    runtime"
    


  • I haven't seen this error before. If you have backups of all the apps, you could try to start with an empty/var/lib/docker folder and restore all apps from the Cloudron dashboard, this would hopefully refetch all images and rebuild the app containers as well.

    Also which Cloudron version are you using and just to be sure that docker was not somehow updated by the system automatically, which docker version is this?



  • I dont update manualy as instructed by cloudron. Luckily i do use digitalocean snapshots before doing this kinda of stuff. I am not good with this stuff now i am scared to try again. My intention was the inform you guys about the problem maybe docs out of date or may be i am missing something. Increased disk size on vm so i dont have do this for now. Thanks for answer.

    Cloudron version : v2.4.1

    root@my:~# docker info
    Containers: 18
     Running: 15
     Paused: 0
     Stopped: 3
    Images: 12
    Server Version: 18.03.1-ce
    Storage Driver: overlay2
     Backing Filesystem: extfs
     Supports d_type: true
     Native Overlay Diff: true
    Logging Driver: journald
    Cgroup Driver: cgroupfs
    Plugins:
     Volume: local
     Network: bridge host macvlan null overlay
     Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
    Swarm: inactive
    Runtimes: runc
    Default Runtime: runc
    Init Binary: docker-init
    containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
    runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
    init version: 949e6fa
    Security Options:
     apparmor
     seccomp
      Profile: default
    Kernel Version: 4.4.0-130-generic
    Operating System: Ubuntu 16.04.3 LTS
    OSType: linux
    Architecture: x86_64
    CPUs: 1
    Total Memory: 1.953GiB
    Name: my.sinanerholding.com.tr
    ID: DTCD:EOBU:MMA7:YA2U:TGRX:IPPE:YG5X:ZVE5:26TX:UZQD:AFKZ:74KS
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): false
    Registry: https://index.docker.io/v1/
    Labels:
    Experimental: false
    Insecure Registries:
     127.0.0.0/8
    Live Restore Enabled: false
    


  • @enesaydogan Indeed something seems off. The error report you pasted seems to complain about btrfs but Cloudron itself does not use btrfs. I am trying this on a DO instance myself now.



  • Just tried this and it works for me.

    Note that I too see the following message in my logs:

    Aug 02 17:29:34 my.smartserver.space dockerd[20759]: time="2018-08-02T17:29:34Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." module=containerd type=io.containerd.snapshotter.v1
    Aug 02 17:29:34 my.smartserver.space dockerd[20759]: time="2018-08-02T17:29:34Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /mnt/volume_sfo2_01/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module=containerd
    Aug 02 17:29:34 my.smartserver.space dockerd[20759]: time="2018-08-02T17:29:34Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /mnt/volume_sfo2_01/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module="containerd/io.containerd.metadata.v1.bolt"
    

    But the above error is harmless. It is just saying that for btrfs plugin we need btrfs volume. This can be ignored since Cloudron does not use btrfs plugin.