How to build (custom) apps using the docker-registry

msbt

@girish does the build service maybe require a restart after changing the credentials?

girish

@msbt it does, yes

msbt

@kallados said in How to build (custom) apps using the docker-registry:

Building docker.kallados.com:20210213-171725-92029c79a

Unrelated to the docker registry: This doesn't look right, you're missing the app-name. You should enter docker.kallados.com/my-app after triggering cloudron build

girish

Ah, good spot! That would indeed be a problem if you don't have the org name and repo name correct when building. You can "reset" the repo name with cloudron build --set-repository. Note that if you are using public docker registry, it should be like username/my-app-repo

kallados

Thanks for Tips. I check i now. Credential has been set correctly (i think). Because login to Docker working fine. I installed also Docker Registry UI. Maybe is really something wrong with name

kallados

@msbt Im not sure, what you mean. Btw. where you mean. I changed some steps now, bit testing... and i have on the Docker "own version" of Kutt Shortener... Bit this image have 727 MB.. Pretty big. But this is maybe normal. I see noe the App in Cloudron Dashboard, but is not running. I get this again from scratch.

root@v220201270340135516:~/shortener# docker build -t docker.kallados.com/shortener .
Sending build context to Docker daemon  122.4kB
Step 1/8 : FROM cloudron/base:2.0.0@sha256:f9fea80513aa7c92fe2e7bf3978b54c8ac5222f47a9a32a7f8833edf0eb5a4f4
 ---> afa4cfc125b4
Step 2/8 : RUN mkdir -p /app/code
 ---> Using cache
 ---> f1457aaf57b4
Step 3/8 : WORKDIR /app/code
 ---> Using cache
 ---> 3b37efec5a77
Step 4/8 : ARG VERSION=2.7.0
 ---> Using cache
 ---> 8a5edb37372d
Step 5/8 : RUN curl -L https://github.com/thedevs-network/kutt/archive/v${VERSION}.tar.gz | tar -xz --strip-components 1 -f - &&     ln -s /app/data/env /app/code/.env
 ---> Using cache
 ---> 3ba0e2ef0058
Step 6/8 : RUN npm install && npm run build
 ---> Using cache
 ---> b94fde788cbf
Step 7/8 : COPY start.sh /app/pkg/
 ---> Using cache
 ---> 505a39d706b1
Step 8/8 : CMD [ "/app/pkg/start.sh" ]
 ---> Using cache
 ---> 387ef59734e6
Successfully built 387ef59734e6
Successfully tagged docker.kallados.com/shortener:latest
root@v220201270340135516:~/shortener# ls
CHANGELOG              DESCRIPTION.md  logo.png        README.md    start.sh
CloudronManifest.json  Dockerfile      POSTINSTALL.md  screenshots  test
root@v220201270340135516:~/shortener# docker push docker.kallados.com/shortener
The push refers to repository [docker.kallados.com/shortener]
4a1ea3b3d2fd: Pushed 
e22aa1fdde33: Pushed 
bf2b3d7dd2c1: Pushed 
544a933d9f36: Pushed 
fcdfeda3e242: Pushed 
0ea3bde29271: Pushed 
d75ccb14b8b6: Pushed 
74b4389a43ab: Pushed 
5f38ae1e1a63: Pushed 
3479c151673d: Pushed 
7a307b866f25: Pushed 
ce3a66c20e17: Pushed 
7197b970ebb9: Pushed 
16542a8fc3be: Pushed 
6597da2e2e52: Pushed 
977183d4e999: Pushed 
c8be1b8f4d60: Pushed 
latest: digest: sha256:eba1b83cc0dddbac09d8ec7fb0bf675887c6a4a57878fbe4b8036e1582e39590 size: 3875
root@v220201270340135516:~/shortener# cloudron install --image docker.kallados.com/shortener
Location: 
root@v220201270340135516:~/shortener# cloudron install --image docker.kallados.com/shortener
Location: shortener.kallados.com
App is being installed.

 => Queued .
 => Registering subdomains ....
 => Setting up addons ..............
 => Creating container 
 => Waiting for DNS propagation .
 => Wait for health check ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C
root@v220201270340135516:~/shortener#

And here Log from App.

Feb 13 21:40:19 box:addons shortener.kallados.com Setting up addon redis with options {}
Feb 13 21:40:19 box:addons Re-using existing redis container with state: {"Status":"exited","Running":false,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":0,"ExitCode":0,"Error":"","StartedAt":"2021-02-13T20:33:01.889401025Z","FinishedAt":"2021-02-13T20:40:09.139054778Z"}
Feb 13 21:40:19 box:addons Waiting for redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb
Feb 13 21:40:19 box:addons Error setting up redis: { BoxError: Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service
at /home/yellowtent/box/src/services.js:275:34
at Object.callback (/home/yellowtent/box/src/docker.js:555:9)
at /home/yellowtent/box/node_modules/dockerode/lib/container.js:73:12
at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:273:7)
at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:232:14)
at IncomingMessage.emit (events.js:203:15)
at endReadableNT (_stream_readable.js:1145:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
name: 'BoxError',
reason: 'Inactive',
details: {},
message:
'Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service' }
Feb 13 21:40:19 box:apptask shortener.kallados.com error creating : BoxError: Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service
Feb 13 21:40:19 box:apptask shortener.kallados.com updating app with values: {"installationState":"error","error":{"message":"Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service","reason":"Inactive","taskId":"58","installationState":"pending_resize"}}
Feb 13 21:40:19 box:taskworker Task took 0.314 seconds
Feb 13 21:40:19 box:tasks setCompleted - 58: {"result":null,"error":{"stack":"BoxError: Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service\n at /home/yellowtent/box/src/services.js:275:34\n at Object.callback (/home/yellowtent/box/src/docker.js:555:9)\n at /home/yellowtent/box/node_modules/dockerode/lib/container.js:73:12\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:273:7)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:232:14)\n at IncomingMessage.emit (events.js:203:15)\n at endReadableNT (_stream_readable.js:1145:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)","name":"BoxError","reason":"Inactive","details":{},"message":"Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service"}}
Feb 13 21:40:19 box:tasks 58: {"percent":100,"result":null,"error":{"stack":"BoxError: Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service\n at /home/yellowtent/box/src/services.js:275:34\n at Object.callback (/home/yellowtent/box/src/docker.js:555:9)\n at /home/yellowtent/box/node_modules/dockerode/lib/container.js:73:12\n at Modem.buildPayload (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:273:7)\n at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/docker-modem/lib/modem.js:232:14)\n at IncomingMessage.emit (events.js:203:15)\n at endReadableNT (_stream_readable.js:1145:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)","name":"BoxError","reason":"Inactive","details":{},"message":"Error getting IP of redis-202151ec-54d4-4c8d-ae7a-2a96d5ae01bb service"}}
Feb 13 21:33:17 ==> First run. Creating env
Feb 13 21:33:17 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:18 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:18 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:19 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:20 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:22 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:26 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:32 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:46 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:34:12 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:35:03 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:36:03 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:37:04 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:38:04 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:39:05 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:40:05 /app/pkg/start.sh: line 15: CLOUDRON_APP_DOMAIN: unbound variable
Feb 13 21:33:01 Generating SSL certificate
Feb 13 21:33:01 Generating a RSA private key
Feb 13 21:33:01 .+++++
Feb 13 21:33:01 ......+++++
Feb 13 21:33:01 writing new private key to '/run/redis.cloudron.key'
Feb 13 21:33:01 -----
Feb 13 21:33:01 Starting supervisor
Feb 13 21:33:02 2021-02-13 20:33:02,019 CRIT Supervisor running as root (no user in config file)
Feb 13 21:33:02 2021-02-13 20:33:02,019 INFO Included extra file "/etc/supervisor/conf.d/redis-service.conf" during parsing
Feb 13 21:33:02 2021-02-13 20:33:02,019 INFO Included extra file "/etc/supervisor/conf.d/redis.conf" during parsing
Feb 13 21:33:02 2021-02-13 20:33:02,027 INFO RPC interface 'supervisor' initialized
Feb 13 21:33:02 2021-02-13 20:33:02,027 CRIT Server 'inet_http_server' running without any HTTP authentication checking
Feb 13 21:33:02 2021-02-13 20:33:02,028 INFO RPC interface 'supervisor' initialized
Feb 13 21:33:02 2021-02-13 20:33:02,028 CRIT Server 'unix_http_server' running without any HTTP authentication checking
Feb 13 21:33:02 2021-02-13 20:33:02,028 INFO supervisord started with pid 1
Feb 13 21:33:03 2021-02-13 20:33:03,030 INFO spawned: 'redis' with pid 15
Feb 13 21:33:03 2021-02-13 20:33:03,032 INFO spawned: 'redis-service' with pid 16
Feb 13 21:33:03 15:C 13 Feb 20:33:03.035 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
Feb 13 21:33:03 15:C 13 Feb 20:33:03.035 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=15, just started
Feb 13 21:33:03 15:C 13 Feb 20:33:03.035 # Configuration loaded
Feb 13 21:33:03 15:M 13 Feb 20:33:03.036 * Running mode=standalone, port=6379.
Feb 13 21:33:03 15:M 13 Feb 20:33:03.036 # Server initialized
Feb 13 21:33:03 15:M 13 Feb 20:33:03.036 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
Feb 13 21:33:03 15:M 13 Feb 20:33:03.036 * Ready to accept connections
Feb 13 21:33:03 Redis service endpoint listening on https://:::3000
Feb 13 21:33:04 2021-02-13 20:33:04,138 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Feb 13 21:33:04 2021-02-13 20:33:04,138 INFO success: redis-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Feb 13 21:33:16 [GET] /healthcheck
Feb 13 21:40:09 2021-02-13 20:40:09,089 WARN received SIGTERM indicating exit request
Feb 13 21:40:09 2021-02-13 20:40:09,089 INFO waiting for redis, redis-service to die
Feb 13 21:40:09 2021-02-13 20:40:09,091 INFO stopped: redis-service (terminated by SIGTERM)
Feb 13 21:40:09 15:signal-handler (1613248809) Received SIGTERM scheduling shutdown...
Feb 13 21:40:09 15:M 13 Feb 20:40:09.109 # User requested shutdown...
Feb 13 21:40:09 15:M 13 Feb 20:40:09.109 * Saving the final RDB snapshot before exiting.
Feb 13 21:40:09 15:M 13 Feb 20:40:09.110 * DB saved on disk
Feb 13 21:40:09 15:M 13 Feb 20:40:09.110 * Removing the pid file.
Feb 13 21:40:09 15:M 13 Feb 20:40:09.110 # Redis is now ready to exit, bye bye...
Feb 13 21:40:09 2021-02-13 20:40:09,111 INFO stopped: redis (exit status 0)

girish

@kallados Ah, I think that is fixed by Services -> Redis -> Restart ? And then come back to this view and Retry resize.

kallados

I can't get it I can build everything is fine. But i can't push it to Docker Registry UI.

Cloudron Build Service: https://builder.kallados.com
Docker Registry UI: https://docker.kallados.com

Git with Original Base Image from Cloudron: https://gitea.kallados.com/root/shortener

Building - OK

root@v220201270340137003:~/shortener# cloudron build --set-build-service
Enter build service URL: builder.kallados.com        
Using build service https://builder.kallados.com
Building kallados/shortener:20210214-180129-524065527
Uploading source tarball...
Build Service login (https://builder.kallados.com):
Username: kallados
Password: ****************************
Login successful.

.....
Compiled successfully.

Push - NOK

Successfully built 40310b79e0c2
Successfully tagged kallados/shortener:20210214-180518-5970157a5
The push refers to repository [docker.io/kallados/shortener]

For some reason... he want to push just in docker.io and not docker.kallados.com

But my docker.json on Builder App is fine...

{
  "docker.kallados.com": {
    "username": "kallados",
    "password": "xxx"
  }
}

I tested few times -> cloudron build --set-repository

But get error (Builder LOG)

Feb 14 19:11:45 build-1613326304870 Building...
Feb 14 19:11:45 build-1613326304870 docker build -t docker.kallados.com/kallados/life.gd.shortener:20210214-181144-759ce076f (Dockerfile)
Feb 14 19:11:45 build-1613326304870 Build stream finished
Feb 14 19:11:45 build-1613326304870 Pushing...
Feb 14 19:12:15 build-1613326304870 has status running. Logs at /tmp/build-1613326304870.log
Feb 14 19:12:35 build-1613326304870 Push stream finished
Feb 14 19:12:35 build-1613326304870 received unexpected HTTP status: 200 OK
Feb 14 19:12:35 build-1613326304870 Push failed
Feb 14 19:12:35 ERROR Build build-1613326304870 failed during image pushing. [ /app/code/src/builds.js:142:25 ]

Maybe is something wrong with my Repository Adress, because this here not working

root@v220201270340137003:~/shortener# cloudron build --set-repository
Enter repository (e.g registry/username/it.kutt.cloudronapp): docker.kallados.com/kallados/life.gd.shortener

Thanks for your support guys

msbt

@kallados did you also restart the build-app after setting the docker stuff?

kallados

This post is deleted!

robi

does the cloudron npm need an update? what version is it?

kallados

@robi

6.14.4

where i run cloudron CLI and 7.5.4 on Cloudron Builder Server. Seems to be same problem here maybe.
https://forum.cloudron.io/topic/4459/cloudron-build-mess-up
Some Issue with Name with new Cloudron Registry UI App

Maybe is here some problem with Cloudron build and subdomain. My Repo is with subdomain docker.kallados.com

cloudron build --set-repository
Enter repository (e.g registry/username/it.kutt.cloudronapp): docker.kallados.com/kallados/short

get Output:

Successfully tagged docker.kallados.com/kallados/short:20210214-211247-622543dc1
The push refers to repository [docker.kallados.com/kallados/short]

End error:

Feb 14 22:12:48 ===========================================
Feb 14 22:12:48 Application Build Service Worker
Feb 14 22:12:48 ===========================================
Feb 14 22:12:48
Feb 14 22:12:48 Docker Username: kallados
Feb 14 22:12:48 Build logfile: /tmp/build-1613337167754.log
Feb 14 22:12:48 Build Id: build-1613337167754
Feb 14 22:12:48 Image Repo: docker.kallados.com/kallados/short
Feb 14 22:12:48 Image Tag: 20210214-211247-622543dc1
Feb 14 22:12:48 Push: true
Feb 14 22:12:48 Source Archive: /tmp/ZHo6Al3DzbPVKU5eUfGpUMN2.gz
Feb 14 22:12:48 Use cache: true
Feb 14 22:12:48 Dockerfile Name: Dockerfile
Feb 14 22:12:48
Feb 14 22:12:48 ===========================================
Feb 14 22:12:48
Feb 14 22:12:48 build-1613337167754 Building...
Feb 14 22:12:48 build-1613337167754 docker build -t docker.kallados.com/kallados/short:20210214-211247-622543dc1 (Dockerfile)
Feb 14 22:12:48 build-1613337167754 Build stream finished
Feb 14 22:12:48 build-1613337167754 Pushing...
Feb 14 22:13:18 build-1613337167754 has status running. Logs at /tmp/build-1613337167754.log
Feb 14 22:13:38 build-1613337167754 Push stream finished
Feb 14 22:13:38 build-1613337167754 received unexpected HTTP status: 200 OK
Feb 14 22:13:38 build-1613337167754 Push failed
Feb 14 22:13:38 ERROR Build build-1613337167754 failed during image pushing. [ /app/code/src/builds.js:142:25 ]

If i use just just

cloudron build --set-repository
Enter repository (e.g registry/username/it.kutt.cloudronapp): kallados/shortener

get answer

Successfully tagged kallados/shortener:20210214-211904-2725dd144
The push refers to repository [docker.io/kallados/shortener]

But docker.io has been nowhere stored from me. docker.json on Builder is correct.

*** I have tried to enter a port. Interesting why IPv6 pops out... but at least I get response from the server.

Successfully built 40310b79e0c2
Successfully tagged gd.life:8000/kallados/short:20210214-221223-9816a45d7
The push refers to repository [gd.life:8000/kallados/short]
Get https://gd.life:8000/v2/: dial tcp [xxx:3549::3]:8000: connect: no route to host
Failed to build app. See log output above.

So i got it successfully and pushed into Registry on Cloudron. I had just to use Docker and not Cloudron Builder.

robi

docker.io is the default, unless overidden with a private registry.

kallados

@robi My privat registry running on docker.kallados.com, unfortunately just with docker. I cab't use Builder for some Reason.

girish

@kallados I think the issue you hit is that 6.1.2 has a bug where pushing from a custom docker client fails (in this case, the docker client is the build service). https://forum.cloudron.io/topic/4383/cannot-install-apps-from-docker-registry-because-authentication-fails/8 has the fix.

plusone-nick

@msbt 🦾+1

BrutalBirdie

@msbt Nice guide!

There is something that I miss tho.
I like to run my cloudron apps local first so I can see if everything is working as intended.
This only works to a certain level for example this can't work if a database addon is used.
But everything before that step can be tested locally.

A full local test suite would be awesome, so I the developer don't have to push every test and deploy it.

For example this is how I do some local testing before I push my image.

Needed software is jq for json parsing, docker, bash, sed

Script explained with words:

Get the ID and version Tag from the CloudronManifest.json and use them for docker build.
Create local folders for /tmp /app/data and /run so I can emulate cloudrons readonly behavior.
Cleanup local folders so every test is fresh and clean
Build the docker image with Data from CloudronManifest.json
Run the freshly build Image in readonly mode with local test folders and an interactive bash session so I can debug / test some stuff.

#!/bin/bash

set -x

ID=$(jq -r ".id" CloudronManifest.json)
VERSION=$(jq -r ".version" CloudronManifest.json')

echo "=> Create Test Data dir"
mkdir -p ./cloudron_test/data ./cloudron_test/tmp ./cloudron_test/run

echo "=> Cleanup Test Data"
rm -rf ./cloudron_test/data/* ./cloudron_test/tmp/* ./cloudron_test/run/*

echo "=> Build test image"
docker build -t dr.cloudron.dev/$ID:$VERSION .

echo "=> Run $VERSION tag of build image of $ID"
docker run -ti --read-only \
    --volume $(pwd)/cloudron_test/data:/app/data:rw \
    --volume $(pwd)/cloudron_test/tmp:/tmp:rw \
    --volume $(pwd)/cloudron_test/run:/run:rw \
    dr.cloudron.dev/$ID:$VERSION \
    bash

infogulch

@brutalbirdie btw, you can get raw strings from jq directly without having to strip quotes with the -r option:

ID=$(jq -r ".id" CloudronManifest.json)
VERSION=$(jq -r ".version" CloudronManifest.json)

This does one better than just stripping quotes since it will also decode other escaped characters (not that that applies in this case).

BrutalBirdie

@infogulch RTFM myself nice catch, thanks for that.

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Cloudron Forum

How to build (custom) apps using the docker-registry

Prerequisites:

How to build apps and push to your Docker-Registry

Optional: use the Cloudron Build Service

With local build

With the Build Service

Prerequisites:

How to build apps and push to your Docker-Registry

Optional: use the Cloudron Build Service

With local build

With the Build Service