Unable to delete custom app - Docker error

[xarp]

Hello,
I attempted to install Zulip as a custom app. It didn't work, however I can't remove the app from the app page (or CLI) due to the error below. I'm getting certificate renewal errors every day.
Any suggestions appreciated.

May 14 10:46:39 box:settings initCache: pre-load settings
May 14 10:46:39 box:taskworker Starting task 606. Logs are at /home/yellowtent/platformdata/logs/6287dbc7-a16c-4930-85d3-2dc713f3e83e/apptask.log
May 14 10:46:39 box:tasks 606: {"percent":2,"error":null}
May 14 10:46:39 box:apptask zulip.xx.com startTask installationState: pending_uninstall runState: running
May 14 10:46:39 box:tasks 606: {"percent":20,"message":"Deleting container"}
May 14 10:46:39 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 14 10:46:39 box:apptask zulip.xx.com deleting app containers (app, scheduler)
May 14 10:46:39 box:shell removeCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh remove 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 14 10:46:39 box:shell removeCollectdProfile (stdout): Restarting collectd
May 14 10:46:41 box:shell removeCollectdProfile (stdout): Removing collectd stats of 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 14 10:46:41 box:shell removeLogrotateConfig spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh remove 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 14 10:46:41 box:apptask zulip.xx.com updating app with values: {"containerId":null}
May 14 10:46:41 box:tasks 606: {"percent":30,"message":"Teardown addons"}
May 14 10:46:41 box:services zulip.xx.com teardownAddons: Tearing down ["localstorage","redis","postgresql"]
May 14 10:46:41 box:services zulip.xx.com Tearing down addon localstorage with options {}
May 14 10:46:41 box:services zulip.xx.com teardownLocalStorage
May 14 10:46:41 box:services zulip.xx.com Tearing down addon redis with options {}
May 14 10:46:41 box:shell removeVolume spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/rmaddondir.sh redis 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 14 10:46:41 box:services zulip.xx.com Tearing down addon postgresql with options {}
May 14 10:46:42 box:tasks 606: {"percent":40,"message":"Cleanup file manager"}
May 14 10:46:42 box:tasks 606: {"percent":50,"message":"Deleting app data directory"}
May 14 10:46:42 box:tasks 606: {"percent":60,"message":"Deleting image"}
May 14 10:46:42 box:docker Error removing image https://hub.docker.com/r/zulip/docker-zulip : {"statusCode":301,"json":{"type":"Buffer","data":[]}}
May 14 10:46:42 box:apptask zulip.xx.com error uninstalling app: BoxError: (HTTP code 301) unexpected -
at Object.callback (/home/yellowtent/box/src/docker.js:591:29)
at /home/yellowtent/box/node_modules/dockerode/lib/image.js:269:12
at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:309:7
at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)
at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)
at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)
at IncomingMessage.emit (events.js:327:22)
at endReadableNT (internal/streams/readable.js:1327:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
reason: 'Docker Error',
details: [Object],
nestedError: [Error]
May 14 10:46:42 box:apptask zulip.xx.com updating app with values: {"installationState":"error","error":{"message":"(HTTP code 301) unexpected - ","statusCode":301,"json":{"type":"Buffer","data":[]},"taskId":"606","installationState":"pending_uninstall"}}
May 14 10:46:42 box:taskworker Task took 2.562 seconds
May 14 10:46:42 box:tasks setCompleted - 606: {"result":null,"error":{"stack":"BoxError: (HTTP code 301) unexpected - \n at Object.callback (/home/yellowtent/box/src/docker.js:591:29)\n at /home/yellowtent/box/node_modules/dockerode/lib/image.js:269:12\n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:309:7\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"BoxError","reason":"Docker Error","details":{},"message":"(HTTP code 301) unexpected - ","nestedError":{"stack":"Error: (HTTP code 301) unexpected - \n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:301:17\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"Error","message":"(HTTP code 301) unexpected - "}}}
May 14 10:46:42 box:tasks 606: {"percent":100,"result":null,"error":{"stack":"BoxError: (HTTP code 301) unexpected - \n at Object.callback (/home/yellowtent/box/src/docker.js:591:29)\n at /home/yellowtent/box/node_modules/dockerode/lib/image.js:269:12\n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:309:7\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"BoxError","reason":"Docker Error","details":{},"message":"(HTTP code 301) unexpected - ","nestedError":{"stack":"Error: (HTTP code 301) unexpected - \n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:301:17\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"Error","message":"(HTTP code 301) unexpected - "}}}

[nebulon]

@xarp looks a bit like a docker daemon error, can you try to restart the docker service from the Cloudron dashboard in the services view, then try to uninstall again.

[xarp]

@nebulon Thanks for the reply. Unfortunately that didn't help. The server has also been rebooted several times since first trying to uninstall.

May 16 14:48:13 box:settings initCache: pre-load settings
May 16 14:48:13 box:taskworker Starting task 638. Logs are at /home/yellowtent/platformdata/logs/6287dbc7-a16c-4930-85d3-2dc713f3e83e/apptask.log
May 16 14:48:13 box:tasks 638: {"percent":2,"error":null}
May 16 14:48:13 box:apptask zulip.xx.com startTask installationState: pending_uninstall runState: running
May 16 14:48:13 box:tasks 638: {"percent":20,"message":"Deleting container"}
May 16 14:48:13 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 16 14:48:14 box:apptask zulip.xx.com deleting app containers (app, scheduler)
May 16 14:48:14 box:shell removeCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh remove 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 16 14:48:14 box:shell removeCollectdProfile (stdout): Restarting collectd
May 16 14:48:14 box:shell removeCollectdProfile (stdout): Removing collectd stats of 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 16 14:48:14 box:shell removeLogrotateConfig spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh remove 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 16 14:48:14 box:apptask zulip.xx.com updating app with values: {"containerId":null}
May 16 14:48:14 box:tasks 638: {"percent":30,"message":"Teardown addons"}
May 16 14:48:14 box:services zulip.xx.com teardownAddons: Tearing down ["localstorage","redis","postgresql"]
May 16 14:48:14 box:services zulip.xx.com Tearing down addon localstorage with options {}
May 16 14:48:14 box:services zulip.xx.com teardownLocalStorage
May 16 14:48:14 box:services zulip.xx.com Tearing down addon redis with options {}
May 16 14:48:14 box:shell removeVolume spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/rmaddondir.sh redis 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 16 14:48:14 box:services zulip.xx.com Tearing down addon postgresql with options {}
May 16 14:48:14 box:tasks 638: {"percent":40,"message":"Cleanup file manager"}
May 16 14:48:14 box:tasks 638: {"percent":50,"message":"Deleting app data directory"}
May 16 14:48:14 box:tasks 638: {"percent":60,"message":"Deleting image"}
May 16 14:48:14 box:docker Error removing image https://hub.docker.com/r/zulip/docker-zulip : {"statusCode":301,"json":{"type":"Buffer","data":[]}}
May 16 14:48:14 box:apptask zulip.xx.com error uninstalling app: BoxError: (HTTP code 301) unexpected -
at Object.callback (/home/yellowtent/box/src/docker.js:591:29)
at /home/yellowtent/box/node_modules/dockerode/lib/image.js:269:12
at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:309:7
at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)
at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)
at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)
at IncomingMessage.emit (events.js:327:22)
at endReadableNT (internal/streams/readable.js:1327:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
reason: 'Docker Error',
details: [Object],
nestedError: [Error]
May 16 14:48:14 box:apptask zulip.xx.com updating app with values: {"installationState":"error","error":{"message":"(HTTP code 301) unexpected - ","statusCode":301,"json":{"type":"Buffer","data":[]},"taskId":"638","installationState":"pending_uninstall"}}
May 16 14:48:14 box:taskworker Task took 1.128 seconds
May 16 14:48:14 box:tasks setCompleted - 638: {"result":null,"error":{"stack":"BoxError: (HTTP code 301) unexpected - \n at Object.callback (/home/yellowtent/box/src/docker.js:591:29)\n at /home/yellowtent/box/node_modules/dockerode/lib/image.js:269:12\n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:309:7\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"BoxError","reason":"Docker Error","details":{},"message":"(HTTP code 301) unexpected - ","nestedError":{"stack":"Error: (HTTP code 301) unexpected - \n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:301:17\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"Error","message":"(HTTP code 301) unexpected - "}}}
May 16 14:48:14 box:tasks 638: {"percent":100,"result":null,"error":{"stack":"BoxError: (HTTP code 301) unexpected - \n at Object.callback (/home/yellowtent/box/src/docker.js:591:29)\n at /home/yellowtent/box/node_modules/dockerode/lib/image.js:269:12\n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:309:7\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"BoxError","reason":"Docker Error","details":{},"message":"(HTTP code 301) unexpected - ","nestedError":{"stack":"Error: (HTTP code 301) unexpected - \n at /home/yellowtent/box/node_modules/docker-modem/lib/modem.js:301:17\n at getCause (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:331:7)\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:300:5)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:275:14)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)","name":"Error","message":"(HTTP code 301) unexpected - "}}}

[nebulon]

@xarp are you able to delete the docker image manually using the docker cli commands?
The command would be something like docker image rm <imageId>

[girish]

@xarp ah, I know this error. This happens because the docker image specified has incorrect syntax/malformed. I reported this upstream a while ago but it looks like docker returns a 301 code when it is supposed to return a 400 (!).

Can you try this:

• Edit the file /home/yellowtent/box/src/docker.js
• Look for the function deleteImage. Just comment out this single line at https://git.cloudron.io/cloudron/box/-/blob/master/src/docker.js#L586

        if (error) {
            debug('Error removing image %s : %j', dockerImage, error);
//            return callback(new BoxError(BoxError.DOCKER_ERROR, error)); <-- comment out this line
        }

• Then , try uninstalling (no need to restart anything).

[girish]

https://github.com/apocas/dockerode/issues/548 was the upstream issue

[xarp]

@girish Thanks for that. I commented out the line in docker.js

        if (error) {
            debug('Error removing image %s : %j', dockerImage, error);
//            return callback(new BoxError(BoxError.DOCKER_ERROR, error));
        }

Upon attempting to uninstall again, this is the error I get:

Error : Task Error - Task 647 crashed with code 1 and signal null

May 17 16:23:45 box:settings initCache: pre-load settings
May 17 16:23:45 box:taskworker Starting task 647. Logs are at /home/yellowtent/platformdata/logs/6287dbc7-a16c-4930-85d3-2dc713f3e83e/apptask.log
May 17 16:23:45 box:tasks 647: {"percent":2,"error":null}
May 17 16:23:45 box:apptask zulip.xx.com startTask installationState: pending_uninstall runState: running
May 17 16:23:45 box:tasks 647: {"percent":20,"message":"Deleting container"}
May 17 16:23:45 box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
May 17 16:23:45 box:apptask zulip.xx.com deleting app containers (app, scheduler)
May 17 16:23:45 box:shell removeCollectdProfile spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurecollectd.sh remove 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 17 16:23:45 box:shell removeCollectdProfile (stdout): Restarting collectd
May 17 16:23:45 box:shell removeCollectdProfile (stdout): Removing collectd stats of 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 17 16:23:45 box:shell removeLogrotateConfig spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh remove 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 17 16:23:45 box:apptask zulip.xx.com updating app with values: {"containerId":null}
May 17 16:23:45 box:tasks 647: {"percent":30,"message":"Teardown addons"}
May 17 16:23:46 box:services zulip.xx.com teardownAddons: Tearing down ["localstorage","redis","postgresql"]
May 17 16:23:46 box:services zulip.xx.com Tearing down addon localstorage with options {}
May 17 16:23:46 box:services zulip.xx.com teardownLocalStorage
May 17 16:23:46 box:services zulip.xx.com Tearing down addon redis with options {}
May 17 16:23:46 box:shell removeVolume spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/rmaddondir.sh redis 6287dbc7-a16c-4930-85d3-2dc713f3e83e
May 17 16:23:46 box:services zulip.xx.com Tearing down addon postgresql with options {}
May 17 16:23:46 box:tasks 647: {"percent":40,"message":"Cleanup file manager"}
May 17 16:23:46 box:tasks 647: {"percent":50,"message":"Deleting app data directory"}
May 17 16:23:46 box:tasks 647: {"percent":60,"message":"Deleting image"}
May 17 16:23:46 box:docker Error removing image https://hub.docker.com/r/zulip/docker-zulip : {"statusCode":301,"json":{"type":"Buffer","data":[]}}
May 17 16:23:46 box:tasks 647: {"percent":70,"message":"Unregistering domains"}
May 17 16:23:46 box:taskworker Error: getaddrinfo ENOTFOUND images
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26)

I see no Zulip image when running sudo docker image ls

[girish]

@xarp it seems the error now is something else - Error: getaddrinfo ENOTFOUND images. Not sure where this is coming from since the subdomain from the logs seems to be zulip. Can you make out what images might be referring to?

[xarp]

@girish I'm really not sure. Seems very generic to me. As indicated above, there's no zulip image in docker images.

[girish]

@xarp Strange, I am unable to make out where the error is coming from. Are you able to send us an email at support@cloudron.io and we can take a look. Should be something minor.

[xarp]

@girish Belated marking as resolved for anybody looking in future. girish did some remote ssh support magic and solved the problem.