cloudron build fais with error: Failed to upload app for building: 400 message: accessToken is missing

[nebulon]

You mean you cannot SSH into the server? What are the errors?

[Daniel-Lobo]

Not that. But there is a terminal you can access fort the app config page that doesn't work for me. I'll try ssh

[Daniel-Lobo]

Can you give me the basics or point me to documentation to connect with ssh?

[nebulon]

Not quite sure where to start then. Your server provider will likely have information on how to connect via SSH, most likely you were connected via SSH during Cloudron installation. If you hadn't done that yourself, maybe ask that person?

For clarification the webterminal from the app config page is only a terminal into the app itself (its docker container) given that the previous test was about the docker host instance, SSH would be useful.

[igaudette]

@nebulon Just jump into the conversation as @Daniel-Lobo is my dev. I installed Cloudron on the server and now we are trying to add a custom app to the platform.

I see that the issue seem to be with Docker registry credentials so I logged to the server via SSH and tried to log in with the same credentials as set in our docker.json file and that worked. see below:

<logs>
root@my:~# docker login
Log in with your Docker ID or email address to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com/ to create one.
You can log in with your password or a Personal Access Token (PAT). Using a limited-scope PAT grants better security and is required for organizations using SSO. Learn more at https://docs.docker.com/go/access-tokens/

Username: xxxxxx
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
</logs>

I tried to ask AI why the credentials weren't working then and it answered me the correct JSON format is the following:

{
"auths": {
"https://index.docker.io/v1/": {
"auth": "base64-encoded-username-and-password"
}
}
}

See ChatGPT chat here: https://chatgpt.com/share/6747192b-ee7c-8005-b402-79b3aac2ef2d

So at this point, is it the format recommended in the Cloudron documentation that is wrong or this format is for Cloudron only and you convert internally to correct JSON structure for docker?

[igaudette]

@nebulon I tried to log in to Docker via the terminal of the build app, and have been able to log in using the web browser one-time code but when it tried to save the credential, we got the read-only issue.

[nebulon]

I think we lost track here. All I was trying to suggest to check host docker instance (which is used via a docker proxy from the build service) if the image exists and if you can push the image from the host. So firstly does that work?

The credentials format and file in the build service app are not the docker configs, but configs for the build service, as such what @Daniel-Lobo had already posted is correct for that.

[Daniel-Lobo]

The credentials on the docker.json file are my cloudron's credentials right? Or docker's?

[nebulon]

So the /app/data/docker.json which resides within the build service app instance, can contain one or many credentials of the docker registries images can be pushed to, if those are private in the first place. https://docs.cloudron.io/apps/docker-builder/#private-registry-auth

Given that the error was during image pushing, I guessed the image was built locally fine and only the pushing to the registry failed. So first thing to ensure is the that the image id points to the correct registry and also setup credentials for that. If you use dockerhub then you have to specify the dockerhub credentials here.

All in all you should only ever need to configure things within the app. The mention of the host docker instance was only to narrow down the issue as we didn't seem to get anywhere.

[Daniel-Lobo]

The images are build, but not pushed. We have the registry app. Our intention is to push to the registry app. With my cloudron's credentials the error was "unauthorized: incorrect username or password". With the docker credentials the error is "denied: requested access to the resource is denied"

[nebulon]

Ok so you do not want to even push to the dockerhub? Then you should specify the custom registry instead of docker.io in the docker.json file with the credentials of that custom registry. Also note that the image repository for that app should then point to the correct registry. You can run cloudron build --set-repository and it will show an example string like registry/username/com.test.cloudronapp here make sure the registry bit is set (otherwise docker falls back to docker.io)

[igaudette]

@nebulon by using our Cloudron credentials and setting our registry URL in the JSON file, it works! However, now we get this error message:

"413 Request Entity Too Large"

Our image make 6.47GB.

[nebulon]

From where is this error coming from? The registry (is that a cloudron app as well) or is it from the buildservice to the cli tool? Do you have more errors on the server side logs?

Also, maybe a wild guess, but since that error is somewhat common when using Cloudflare proxying, if you use that, please disable that first.

[igaudette]

@nebulon When we push the image to the Docker registry app, here is what the build app logs are saying:

Nov 27 10:21:23 Application Build Service Worker
Nov 27 10:21:23 Build Id: build-1732720882415
Nov 27 10:21:23 Build logfile: /tmp/build-1732720882415.log
Nov 27 10:21:23 Docker Username: *********
Nov 27 10:21:23 Dockerfile Name: Dockerfile
Nov 27 10:21:23 Image Repo: registry.//app
Nov 27 10:21:23 Image Tag: 20241127-152056-15345a770
Nov 27 10:21:23 Push: true
Nov 27 10:21:23 Source Archive: /tmp/OtULoDB64vB7KxXP8npFeXVT.gz
Nov 27 10:21:23 Use cache: true
Nov 27 10:21:23 build-1732720882415 Building...
Nov 27 10:21:23 build-1732720882415 docker build -t registry.//app:20241127-152056-15345a770 (Dockerfile)
Nov 27 10:21:26 build-1732720882415 Build stream finished
Nov 27 10:21:26 build-1732720882415 Pushing...
Nov 27 10:25:29 2024-11-27T15:25:29Z
Nov 27 10:25:29 </body>
Nov 27 10:25:29 </html>
Nov 27 10:25:29 <body>
Nov 27 10:25:29 <center><h1>413 Request Entity Too Large</h1></center>
Nov 27 10:25:29 <head><title>413 Request Entity Too Large</title></head>
Nov 27 10:25:29 <hr><center>cloudflare</center>
Nov 27 10:25:29 Build build-1732719728695 failed during image pushing.
Nov 27 10:25:29 build-1732719728695 Push failed
Nov 27 10:25:29 build-1732719728695 Push stream finished
Nov 27 10:25:29 build-1732719728695 unknown: <html>
Nov 27 10:34:59 startLogStream: client closed
Nov 27 10:34:59 startLogStream: finished with undefined
Nov 27 10:44:22 2024-11-27T15:44:22Z
Nov 27 10:44:22 </body>
Nov 27 10:44:22 </html>
Nov 27 10:44:22 <body>
Nov 27 10:44:22 <center><h1>413 Request Entity Too Large</h1></center>
Nov 27 10:44:22 <head><title>413 Request Entity Too Large</title></head>
Nov 27 10:44:22 <hr><center>cloudflare</center>
Nov 27 10:44:22 Build build-1732720882415 failed during image pushing.
Nov 27 10:44:22 build-1732720882415 Push failed
Nov 27 10:44:22 build-1732720882415 Push stream finished
Nov 27 10:44:22 build-1732720882415 unknown: <HTML>

The build and docker registry app are on the same Cloudron server. I just saw for Cloudflare in logs.. currently proxy is disabled for registry but not for build. Do you think it could be related?

[nebulon]

Right, Cloudflare proxying causes lots of side-effects and it has a size limit, so anything going through that above the size threshold will fail. So unless you are hosting a rather static website, it is generally not a great idea to use Cloudflare proxying.

[igaudette]

@nebulon I tried disabling proxy on the build and registry app... and I'm still getting the "413 Request Entity Too Large" error

How it's possible if both apps have proxying disabled? Is it because my.mydomain.com (Cloudron) is proxyfied?

[nebulon]

As mentioned, this was just a wild guess, could be other factors in play here also. Have you tried to rule out some more parts, like only using the buildservice to push to dockerhub instead of your custom registry? The error pretty much hints at some issue with the registry you are using. It would certainly help if you could provide more logging info on that side as well. Also maybe increase memory limit for that?

[Daniel-Lobo]

Thank your your assistance, @nebulon. We're finally able to build and install the app.

[nebulon]

That is great news, what was the issue in the end?

[Daniel-Lobo]

The wrong registry in the docker.json file and cloudfare's "proxying"