cloudron build fais with error: Failed to upload app for building: 400 message: accessToken is missing

[nebulon]

You are on the right path at least, the error indicates that the build service app does not have valid credentials to push to the specified registry. Maybe also check the image ID you are building, if that may contain some other domain to push to. Also check the logs of the build service app and just to make sure, have you restarted the build service app after changing the docker.json?

[Daniel-Lobo]

I restarted it and still the problem.
Looking at the logs I can see the error is intact when pushing the image to the registry:

Build stream finished
Pushing...
failed during image pushing.
Push failed

My docker.json file looks like this:
{
"docker.io": {
"username": "?????",
"password": "?????"
}
}

Is there any configuration I might be missing, perhaps in the Registry app?

[nebulon]

Looking good. Can you replace docker.io with index.docker.io and check again?

[Daniel-Lobo]

Same thing

[nebulon]

I am a bit out of ideas now. Since the build service uses the host docker instance, you could also see if you can SSH into the server and attempt a docker push .... of those images there. Also check if the images are known locally, which they should using docker images. Just to narrow down potential issues.

[Daniel-Lobo]

Well. That another problem. The host terminal doesn't work for me

[nebulon]

You mean you cannot SSH into the server? What are the errors?

[Daniel-Lobo]

Not that. But there is a terminal you can access fort the app config page that doesn't work for me. I'll try ssh

[Daniel-Lobo]

Can you give me the basics or point me to documentation to connect with ssh?

[nebulon]

Not quite sure where to start then. Your server provider will likely have information on how to connect via SSH, most likely you were connected via SSH during Cloudron installation. If you hadn't done that yourself, maybe ask that person?

For clarification the webterminal from the app config page is only a terminal into the app itself (its docker container) given that the previous test was about the docker host instance, SSH would be useful.

[igaudette]

@nebulon Just jump into the conversation as @Daniel-Lobo is my dev. I installed Cloudron on the server and now we are trying to add a custom app to the platform.

I see that the issue seem to be with Docker registry credentials so I logged to the server via SSH and tried to log in with the same credentials as set in our docker.json file and that worked. see below:

<logs>
root@my:~# docker login
Log in with your Docker ID or email address to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com/ to create one.
You can log in with your password or a Personal Access Token (PAT). Using a limited-scope PAT grants better security and is required for organizations using SSO. Learn more at https://docs.docker.com/go/access-tokens/

Username: xxxxxx
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
</logs>

I tried to ask AI why the credentials weren't working then and it answered me the correct JSON format is the following:

{
"auths": {
"https://index.docker.io/v1/": {
"auth": "base64-encoded-username-and-password"
}
}
}

See ChatGPT chat here: https://chatgpt.com/share/6747192b-ee7c-8005-b402-79b3aac2ef2d

So at this point, is it the format recommended in the Cloudron documentation that is wrong or this format is for Cloudron only and you convert internally to correct JSON structure for docker?

[igaudette]

@nebulon I tried to log in to Docker via the terminal of the build app, and have been able to log in using the web browser one-time code but when it tried to save the credential, we got the read-only issue.

[nebulon]

I think we lost track here. All I was trying to suggest to check host docker instance (which is used via a docker proxy from the build service) if the image exists and if you can push the image from the host. So firstly does that work?

The credentials format and file in the build service app are not the docker configs, but configs for the build service, as such what @Daniel-Lobo had already posted is correct for that.

[Daniel-Lobo]

The credentials on the docker.json file are my cloudron's credentials right? Or docker's?

[nebulon]

So the /app/data/docker.json which resides within the build service app instance, can contain one or many credentials of the docker registries images can be pushed to, if those are private in the first place. https://docs.cloudron.io/apps/docker-builder/#private-registry-auth

Given that the error was during image pushing, I guessed the image was built locally fine and only the pushing to the registry failed. So first thing to ensure is the that the image id points to the correct registry and also setup credentials for that. If you use dockerhub then you have to specify the dockerhub credentials here.

All in all you should only ever need to configure things within the app. The mention of the host docker instance was only to narrow down the issue as we didn't seem to get anywhere.

[Daniel-Lobo]

The images are build, but not pushed. We have the registry app. Our intention is to push to the registry app. With my cloudron's credentials the error was "unauthorized: incorrect username or password". With the docker credentials the error is "denied: requested access to the resource is denied"

[nebulon]

Ok so you do not want to even push to the dockerhub? Then you should specify the custom registry instead of docker.io in the docker.json file with the credentials of that custom registry. Also note that the image repository for that app should then point to the correct registry. You can run cloudron build --set-repository and it will show an example string like registry/username/com.test.cloudronapp here make sure the registry bit is set (otherwise docker falls back to docker.io)

[igaudette]

@nebulon by using our Cloudron credentials and setting our registry URL in the JSON file, it works! However, now we get this error message:

"413 Request Entity Too Large"

Our image make 6.47GB.

[nebulon]

From where is this error coming from? The registry (is that a cloudron app as well) or is it from the buildservice to the cli tool? Do you have more errors on the server side logs?

Also, maybe a wild guess, but since that error is somewhat common when using Cloudflare proxying, if you use that, please disable that first.

[igaudette]

@nebulon When we push the image to the Docker registry app, here is what the build app logs are saying:

Nov 27 10:21:23 Application Build Service Worker
Nov 27 10:21:23 Build Id: build-1732720882415
Nov 27 10:21:23 Build logfile: /tmp/build-1732720882415.log
Nov 27 10:21:23 Docker Username: *********
Nov 27 10:21:23 Dockerfile Name: Dockerfile
Nov 27 10:21:23 Image Repo: registry.//app
Nov 27 10:21:23 Image Tag: 20241127-152056-15345a770
Nov 27 10:21:23 Push: true
Nov 27 10:21:23 Source Archive: /tmp/OtULoDB64vB7KxXP8npFeXVT.gz
Nov 27 10:21:23 Use cache: true
Nov 27 10:21:23 build-1732720882415 Building...
Nov 27 10:21:23 build-1732720882415 docker build -t registry.//app:20241127-152056-15345a770 (Dockerfile)
Nov 27 10:21:26 build-1732720882415 Build stream finished
Nov 27 10:21:26 build-1732720882415 Pushing...
Nov 27 10:25:29 2024-11-27T15:25:29Z
Nov 27 10:25:29 </body>
Nov 27 10:25:29 </html>
Nov 27 10:25:29 <body>
Nov 27 10:25:29 <center><h1>413 Request Entity Too Large</h1></center>
Nov 27 10:25:29 <head><title>413 Request Entity Too Large</title></head>
Nov 27 10:25:29 <hr><center>cloudflare</center>
Nov 27 10:25:29 Build build-1732719728695 failed during image pushing.
Nov 27 10:25:29 build-1732719728695 Push failed
Nov 27 10:25:29 build-1732719728695 Push stream finished
Nov 27 10:25:29 build-1732719728695 unknown: <html>
Nov 27 10:34:59 startLogStream: client closed
Nov 27 10:34:59 startLogStream: finished with undefined
Nov 27 10:44:22 2024-11-27T15:44:22Z
Nov 27 10:44:22 </body>
Nov 27 10:44:22 </html>
Nov 27 10:44:22 <body>
Nov 27 10:44:22 <center><h1>413 Request Entity Too Large</h1></center>
Nov 27 10:44:22 <head><title>413 Request Entity Too Large</title></head>
Nov 27 10:44:22 <hr><center>cloudflare</center>
Nov 27 10:44:22 Build build-1732720882415 failed during image pushing.
Nov 27 10:44:22 build-1732720882415 Push failed
Nov 27 10:44:22 build-1732720882415 Push stream finished
Nov 27 10:44:22 build-1732720882415 unknown: <HTML>

The build and docker registry app are on the same Cloudron server. I just saw for Cloudflare in logs.. currently proxy is disabled for registry but not for build. Do you think it could be related?