LocalAI platform

[james]

Hello @eyecreate
Apologies, I have simply missed your topic somehow.
I have now added your community app to https://forum.cloudron.io/topic/15172/community-apps

[timconsidine]

@eyecreate well done, thank you, presumably it needs GPU to work efficiently ?

[eyecreate]

I haven't tested it with a gpu(would need to pass device through i think), but yeah, it likely won't work well without one(though ik_llama backend might be good for modern cpus).

[max]

Hi @eyecreate I have installed your localai app, all went well.
I have one issue when I try to connect my LibreChat to it - where do I change the authentication key?

If I test it with I get an expected auth error...
$ curl https://localai.xxxx.net/v1/models
{"error":{"code":401,"message":"An authentication key is required","type":"invalid_request_error"}}

[max]

OK, I think I have worked it out - use the Cloudon CLI...
cloudron env set --app localai.mydomain.com LOCALAI_API_KEY=xxxx
Thanks

[timconsidine]

@eyecreate I tried the community app but everything is just 502.
But I also tried installing their Mac DMG and it could not work with the stated/recommended localhost:8080.

I think it is broken upstream.

Their webpage says run models on your CPU, don't need GPU. Well, it doesn't run on 128Gb RAM modestly used by other apps.

[eyecreate]

Yeah, I've had to report bugs sometimes on hardware compatibility issues.
I've gotten rocm backends working for llama.cpp on the community app, but not others(and something seems off performance wise).

[micmc]

Hi @eyecreate Apparently now the image is not available anymore from the docker repository?
For the moment it refuses to install even after a few retries.

An error occurred during the install operation: Not found: Unable to pull image docker.io/eyecreate/cloudron-localai:v4.10.0. message: (HTTP code 404) unexpected - manifest for eyecreate/cloudron-localai:v4.10.0 not found: manifest unknown: manifest unknown statusCode: 404

Are you still maintaining this package?
Thanks

[micmc]

@eyecreate Hi, I'm not sure if the app has been updated because of my previous post, nevertheless it's still not installing but this time I get a different error message when I try to install it.

Oct 07 15:21:35 BoxError: Unable to pull image registry.eyecreate.org/eyecreate/cloudron-localai:20261006-143345-538b9ad22. Please check the network or if the image needs authentication. statusCode: 401
    at pullImage (file:///home/yellowtent/box/src/docker.js:113:22)
    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
    at async file:///home/yellowtent/box/src/docker.js:483:122
    at async retry (file:///home/yellowtent/box/src/retry.js:13:20)
    at async Object.downloadImage (file:///home/yellowtent/box/src/docker.js:481:5)
    at async downloadImage (file:///home/yellowtent/box/src/apptask.js:197:5)
    at async installCommand (file:///home/yellowtent/box/src/apptask.js:442:5)
Oct 07 15:21:35 Exiting with code 0
Oct 07 15:21:35 taskworker: Task took 62.969 seconds

[eyecreate]

cloudron changed how the cli builds app, so it moved to my registry by default. I had forgotten to make the localai image public, so thanks for pointing it out. Should be fixed now.