Error: listen EADDRNOTAVAIL: address not available 172.18.0.1:3003

[Nozy]

Hi All

1st time I have to jump on be runing good for over 2 year

OK so what I have after a reboot cloudron stop running all show up as no running

local website 1/2 but no service is running

server is up but after running

I get this error over and over I think docker not starting right ?

in the logs
2023-01-25T03:39:18.524Z box:server ==========================================
2023-01-25T03:39:18.526Z box:server Cloudron 7.3.4
2023-01-25T03:39:18.526Z box:server ==========================================
2023-01-25T03:39:18.686Z box:settings initCache: pre-load settings
2023-01-25T03:39:18.731Z box:tasks stopAllTasks: stopping all tasks
2023-01-25T03:39:18.732Z box:shell stopTask spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
2023-01-25T03:39:18.801Z box:shell stopTask (stdout): All tasks stopped

2023-01-25T03:39:18.807Z box:reverseproxy writeDashboardConfig: writing admin config for mozysswamp.com
Cloudron is up and running. Logs are at /home/yellowtent/platformdata/logs/box.log
2023-01-25T03:39:18.896Z box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
2023-01-25T03:39:19.024Z box:cloudron onActivated: running post activation tasks
2023-01-25T03:39:19.025Z box:platform initializing platform
2023-01-25T03:39:19.026Z box:platform platform is uptodate at version 49.4.0
2023-01-25T03:39:19.027Z box:platform onPlatformReady: platform is ready. infra changed: false
2023-01-25T03:39:19.027Z box:apps schedulePendingTasks: scheduling app tasks
2023-01-25T03:39:19.055Z box:cron startJobs: starting cron jobs with hour 2 and minute 2
2023-01-25T03:39:19.110Z box:cron backupConfigChanged: schedule 00 00 1,2,3 * * * (Australia/Melbourne)
2023-01-25T03:39:19.129Z box:cron autoupdatePatternChanged: pattern - 00 00 1,2,3,4,5,23 * * * (Australia/Melbourne)
2023-01-25T03:39:19.134Z box:cron Dynamic DNS setting changed to false
2023-01-25T03:39:19.137Z box:dockerproxy startDockerProxy: started proxy on port 3003
Error: listen EADDRNOTAVAIL: address not available 172.18.0.1:3003
at Server.setupListenHandle [as _listen2] (node:net:1446:21)
at listenInCluster (node:net:1511:12)
at doListen (node:net:1660:7)
at processTicksAndRejections (node:internal/process/task_queues:84:21)

[girish]

@Nozy said in Error: listen EADDRNOTAVAIL: address not available 172.18.0.1:3003:

2023-01-25T03:39:19.137Z box:dockerproxy startDockerProxy: started proxy on port 3003
Error: listen EADDRNOTAVAIL: address not available 172.18.0.1:3003

Something else is running port 3003. Can you try lsof -i :3003 . That should give you the pid of the process that is using that port. Did you happen to install anything outside of Cloudron on the server?

You can also check systemctl status docker to see if docker is running and also docker ps to check if the containers are running.

[Nozy]

@girish

she dead Jim =( ..... it look like docker is dead

[Nozy]

@girish Thanks make for the help too

ok did a $sudo /usr/bin/dockerd -H unix://

web UI is up

root@cloudron:/var/log# sudo /usr/bin/dockerd -H unix://
UI is up and I did remove GM that look like it is looping as it is back and I am sure I had remove it some time ago ( some have start but now all )

hope the logs below help

root@cloudron:/var/log# sudo /usr/bin/dockerd -H unix://
INFO[2023-01-27T02:47:20.595427889Z] Starting up
INFO[2023-01-27T02:47:20.625133706Z] libcontainerd: started new containerd process pid=1304492
INFO[2023-01-27T02:47:20.625354108Z] parsed scheme: "unix" module=grpc
INFO[2023-01-27T02:47:20.625476009Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2023-01-27T02:47:20.625630411Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2023-01-27T02:47:20.625736512Z] ClientConn switching balancer to "pick_first" module=grpc
WARN[0000] containerd config version 1 has been deprecated and will be removed in containerd v2.0, please switch to version 2, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header
INFO[2023-01-27T02:47:20.705971468Z] starting containerd revision=770bd0108c32f3fb5c73ae1264f7e503fe7b2661 version=1.6.10
INFO[2023-01-27T02:47:20.749970137Z] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[2023-01-27T02:47:20.759927143Z] loading plugin "io.containerd.snapshotter.v1.aufs"... type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.766717616Z] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.767454123Z] skip loading plugin "io.containerd.snapshotter.v1.btrfs"... error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.767646125Z] loading plugin "io.containerd.snapshotter.v1.devmapper"... type=io.containerd.snapshotter.v1
WARN[2023-01-27T02:47:20.767832327Z] failed to load plugin io.containerd.snapshotter.v1.devmapper error="devmapper not configured"
INFO[2023-01-27T02:47:20.767936429Z] loading plugin "io.containerd.snapshotter.v1.native"... type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.768828238Z] loading plugin "io.containerd.snapshotter.v1.overlayfs"... type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.770524956Z] loading plugin "io.containerd.snapshotter.v1.zfs"... type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.771202263Z] skip loading plugin "io.containerd.snapshotter.v1.zfs"... error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2023-01-27T02:47:20.771348965Z] loading plugin "io.containerd.metadata.v1.bolt"... type=io.containerd.metadata.v1
WARN[2023-01-27T02:47:20.771566367Z] could not use snapshotter devmapper in metadata plugin error="devmapper not configured"
INFO[2023-01-27T02:47:20.771691169Z] metadata content store policy set policy=shared
INFO[2023-01-27T02:47:20.801519087Z] loading plugin "io.containerd.differ.v1.walking"... type=io.containerd.differ.v1
INFO[2023-01-27T02:47:20.801686388Z] loading plugin "io.containerd.event.v1.exchange"... type=io.containerd.event.v1
INFO[2023-01-27T02:47:20.801815190Z] loading plugin "io.containerd.gc.v1.scheduler"... type=io.containerd.gc.v1
INFO[2023-01-27T02:47:20.802069193Z] loading plugin "io.containerd.service.v1.introspection-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802354096Z] loading plugin "io.containerd.service.v1.containers-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802486797Z] loading plugin "io.containerd.service.v1.content-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802610598Z] loading plugin "io.containerd.service.v1.diff-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802735100Z] loading plugin "io.containerd.service.v1.images-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802788700Z] loading plugin "io.containerd.service.v1.leases-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802872501Z] loading plugin "io.containerd.service.v1.namespaces-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.802993802Z] loading plugin "io.containerd.service.v1.snapshots-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.803119304Z] loading plugin "io.containerd.runtime.v1.linux"... type=io.containerd.runtime.v1
INFO[2023-01-27T02:47:20.803445607Z] loading plugin "io.containerd.runtime.v2.task"... type=io.containerd.runtime.v2
INFO[2023-01-27T02:47:20.803710910Z] loading plugin "io.containerd.monitor.v1.cgroups"... type=io.containerd.monitor.v1
INFO[2023-01-27T02:47:20.805011724Z] loading plugin "io.containerd.service.v1.tasks-service"... type=io.containerd.service.v1
INFO[2023-01-27T02:47:20.805385928Z] loading plugin "io.containerd.grpc.v1.introspection"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.805538230Z] loading plugin "io.containerd.internal.v1.restart"... type=io.containerd.internal.v1
INFO[2023-01-27T02:47:20.811666895Z] loading plugin "io.containerd.grpc.v1.containers"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.811818996Z] loading plugin "io.containerd.grpc.v1.content"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.812092399Z] loading plugin "io.containerd.grpc.v1.diff"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.812234401Z] loading plugin "io.containerd.grpc.v1.events"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.812434203Z] loading plugin "io.containerd.grpc.v1.healthcheck"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.812565404Z] loading plugin "io.containerd.grpc.v1.images"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.812738706Z] loading plugin "io.containerd.grpc.v1.leases"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.812876808Z] loading plugin "io.containerd.grpc.v1.namespaces"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.813052310Z] loading plugin "io.containerd.internal.v1.opt"... type=io.containerd.internal.v1
INFO[2023-01-27T02:47:20.817716259Z] loading plugin "io.containerd.grpc.v1.snapshots"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.817974762Z] loading plugin "io.containerd.grpc.v1.tasks"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.818188564Z] loading plugin "io.containerd.grpc.v1.version"... type=io.containerd.grpc.v1
INFO[2023-01-27T02:47:20.818472367Z] loading plugin "io.containerd.tracing.processor.v1.otlp"... type=io.containerd.tracing.processor.v1
INFO[2023-01-27T02:47:20.818609569Z] skip loading plugin "io.containerd.tracing.processor.v1.otlp"... error="no OpenTelemetry endpoint: skip plugin" type=io.containerd.tracing.processor.v1
INFO[2023-01-27T02:47:20.818730470Z] loading plugin "io.containerd.internal.v1.tracing"... type=io.containerd.internal.v1
ERRO[2023-01-27T02:47:20.818894172Z] failed to initialize a tracing processor "otlp" error="no OpenTelemetry endpoint: skip plugin"
INFO[2023-01-27T02:47:20.819725581Z] serving... address=/var/run/docker/containerd/containerd-debug.sock
INFO[2023-01-27T02:47:20.820064084Z] serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
INFO[2023-01-27T02:47:20.820375388Z] serving... address=/var/run/docker/containerd/containerd.sock
INFO[2023-01-27T02:47:20.820558690Z] containerd successfully booted in 0.127464s
INFO[2023-01-27T02:47:20.999818201Z] parsed scheme: "unix" module=grpc
INFO[2023-01-27T02:47:20.999956603Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2023-01-27T02:47:21.000076004Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2023-01-27T02:47:21.000183405Z] ClientConn switching balancer to "pick_first" module=grpc
INFO[2023-01-27T02:47:21.023866158Z] parsed scheme: "unix" module=grpc
INFO[2023-01-27T02:47:21.024011559Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2023-01-27T02:47:21.024063760Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2023-01-27T02:47:21.024088660Z] ClientConn switching balancer to "pick_first" module=grpc
INFO[2023-01-27T02:47:22.692598335Z] [graphdriver] using prior storage driver: overlay2
WARN[2023-01-27T02:47:32.198713192Z] Your kernel does not support CPU realtime scheduler
WARN[2023-01-27T02:47:32.198854993Z] Your kernel does not support cgroup blkio weight
WARN[2023-01-27T02:47:32.198936494Z] Your kernel does not support cgroup blkio weight_device
INFO[2023-01-27T02:47:32.209024301Z] Loading containers: start.
ERRO[2023-01-27T02:47:33.079163898Z] e20d43d6033f0626a2bea17e82a0a5a6a8c7e796ebafd5fec4ea60543928eaac cleanup: failed to delete container from containerd: no such container
ERRO[2023-01-27T02:47:33.096909785Z] 9d1753e0824b48f7b503605f7e0cfe9332505ce73b829aa068dce830e91c15d7 cleanup: failed to delete container from containerd: no such container
ERRO[2023-01-27T02:47:40.483088915Z] Could not add route to IPv6 network fd00:c107:d509::1/64 via device br-5dd03994b0fe: network is down
INFO[2023-01-27T02:47:41.279535282Z] Removing stale sandbox 2944471fa7e66fbb9623b0b0f978777d06bf63734d7c3df26f6f324f38a48a6c (eb8f18f5060d7f77ac04837d43baa8f8f4a1395eeb690425169c5e3487233356)
WARN[2023-01-27T02:47:41.447723648Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 4d9eb8f3b634d6806b39a0409004c6b4404d0d2bf24fb4a1fb944d87ff10c32c], retrying....
INFO[2023-01-27T02:47:41.884874839Z] Removing stale sandbox c1291f9cf56d54c7d61611ee79ba9ea56882d159ed94710d49a513ad8cc7e821 (c5c06f559a4fa7d674793a42b7f9f9aa77de41043396af298a54b97d4c506bd2)
INFO[2023-01-27T02:47:42.345514474Z] Removing stale sandbox f76db226e86a424feadf10ad5b24ab8883908803326fa1ca04c7767420726475 (7a818c50dae2556b5906890996d706b27a36a10fb3aaad2904d88d0fb952e7f1)
WARN[2023-01-27T02:47:42.435529518Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 beffee3d8c6346f35639be599abd5369ce390a0fe5b60541e37871b6c126c62c], retrying....
INFO[2023-01-27T02:47:42.887012356Z] Removing stale sandbox 02a0083eb84d0488460026afaa03b99e66e248804ea275b0611782477da5f00c (9d1753e0824b48f7b503605f7e0cfe9332505ce73b829aa068dce830e91c15d7)
INFO[2023-01-27T02:47:43.237811035Z] Removing stale sandbox 1db79153ecda3d9e395b7346d27fdab2bbb8b8247481354fe4be31e0cfdc3010 (60d2bf6aeed7c371a9ceff460c3007283c83bdcc455f863795a860edcadcf4ab)
INFO[2023-01-27T02:47:43.710429691Z] Removing stale sandbox 63cde8e1972369eaf955096353e8155697f661b0ac5bb7231ce2074d41087438 (a9d4c2900b96206dac0980015573060f94ea34d3be283127da03706132db28c0)
WARN[2023-01-27T02:47:43.812808165Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 7636a0a1e47b2478e743a53d0cd891e6631f34042eee55a349c6e5177f457ff2], retrying....
INFO[2023-01-27T02:47:44.277778938Z] Removing stale sandbox 63f08690c9f8c0b05be9c90f463bb5ad4e2f5fe8164007b72009c7dc05661a55 (1c0c8c9b53e1c7af0f2d3796a6d7392931c8aa3aea0e292fcc1142e1a8e0ee22)
WARN[2023-01-27T02:47:44.428240615Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 274e234a80c2733025f3fafe61e3bb407464fb053726950ff9719c88df49e010], retrying....
INFO[2023-01-27T02:47:44.925008820Z] Removing stale sandbox 69695305fd378a9e8bf4340f08b7a3d8dae9e3dd7a80ff749cdb10e4be9fcc25 (ead8bf3a118aa51be73b8baf142f9ab38a80912e2ebd9573e90e9c18c51751b6)
WARN[2023-01-27T02:47:45.190151897Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 1780e407a29129998976d25f41c6aa2d7757e589249a576301d02af63c00c5fe], retrying....
INFO[2023-01-27T02:47:45.690266834Z] Removing stale sandbox 6a2b5dcc11c0ce7c029ebdbd7b34455941d3a6b59ba1af499b798c6785e6b693 (b4245017d281b2e2ca0bc36219259115140b45bc6cc7b871a99ec876f4d701ed)
WARN[2023-01-27T02:47:45.884789970Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 54b38d64a6cde3995af72392038547f8d901d691e5c573503717df28c1b7a97d], retrying....
INFO[2023-01-27T02:47:46.356250004Z] Removing stale sandbox 9b4988b1706dce36a96687667173a73ffd9a4d94f75254b5ec236e1266194270 (f892619a77d6e9eb78b1d16a032acc919de3a375eeaca57c3648f686c71b0519)
INFO[2023-01-27T02:47:46.804818798Z] Removing stale sandbox af50ce9f0aeeada25fbcd920214c91ac1dadf675891e8b934ed8d7b688765da3 (85c41d35d3042ddca260ff8636f2e1c84a72db16db0d3f73f21547333d0b1092)
WARN[2023-01-27T02:47:46.914580946Z] Error (Unable to complete atomic operation, key modified) deleting object [endpoint 5dd03994b0fedaeff8fb95498caedc929d6aa9bf4323ae95a6e61fe6c8af5ab3 017f917bcf6acfd806fcc092669004e5a56bf5248d25a1f7e5314be4766d0536], retrying....

[girish]

Thanks for the update! Do you know why it died in the first place?

[Nozy]

@girish just did a reboot have not done one for a bit form what I can see the the gm docker still is not remove even after I did remove it so I think it looping on that

let me get this week thru and I will reboot see how I go
( MAKE SURE I DO BACKUP 1st yes PPL do backups )

[girish]

@Nozy said in Error: listen EADDRNOTAVAIL: address not available 172.18.0.1:3003:

the gm docker still is not remove

If you click on the app icon, it will show you the error as to why it's not uninstalling.