AI on Cloudron

[LoudLemur]

Use AI to fix github issues - SWE-agent:
Docker: Yes
Licence: MIT
https://github.com/princeton-nlp/SWE-agent
https://inv.tux.pizza/watch?v=9-JBHGlYEBI

[LoudLemur]

Non-Free Google Gemini 1.5 pro available
https://threadreaderapp.com/thread/1778026405793828986.html

[marcusquinn]

Apt read on AI: https://www.linkedin.com/posts/paddy-ryan_this-is-an-image-you-may-have-seen-doing-activity-7185881126001897472-vKlK/

[LoudLemur]

Anthropic AI's CEO has said that AI might be able to survive in the wild by next year. It was speculated elsewhere that it could fund itself through OnlyFans.

https://futurism.com/the-byte/anthropic-ceo-ai-replicate-survive

[LoudLemur]

M$FT VASA-1 - Talking Heads

https://www.microsoft.com/en-us/research/project/vasa-1/

[LoudLemur]

Cool interview on progress towards autonomous AI:
https://inv.tux.pizza/watch?v=6RUR6an5hOY&quality=dash

via Arya ai:

Enclosure in the context of technology, particularly open-source, refers to the phenomenon where a technology that was initially open and accessible becomes controlled or monopolized by a specific entity or group. This can happen through various means, such as strategic acquisitions, licensing restrictions, or the development of proprietary features that are not available in the open-source version. Some examples of enclosure include:
Oracle's acquisition of Sun Microsystems, which led to the discontinuation of the open-source OpenSolaris operating system.
Google's acquisition of Android and the development of proprietary features and services (like Google Play Services) that are not available in the open-source version (AOSP).
Microsoft's acquisition of GitHub, which raised concerns about the future of open-source projects hosted on the platform.
These examples illustrate how the enclosure can happen through strategic acquisitions and the development of proprietary features, leading to the control or monopolization of open-source technology.

[LoudLemur]

This tool is good and there is an application request.

https://anythingllm.com
https://forum.cloudron.io/topic/11722/anythingllm-ai-business-intelligence-tool/5

[LoudLemur]

Model strong at coding/bug-fixing:
https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct

[LoudLemur]

OpenSora (not Open AI) the Free Software AI video creator is now available for self-hosting.

https://github.com/hpcaitech/Open-Sora

From Reddit:

"It can generate up to 16 seconds in 1280x720 resolution but requires 67g vram and takes 10minutes to generate on a 80G H100 graphics card which costs 30k. However there are hourly services and I see one that is 3 dollars per hour which is like 50cents per video at the highest rez. So you could technically output a feature length movie (60minutes) with $100.

*Disclaimer: it says minimum requirement is 24g vram, so not going to be easy to run this to its full potential yet.

They do also have a gradio demo as well."

[LoudLemur]

From Reddit:
Etched is working on a specialized AI chip called Sohu. Unlike general-purpose GPUs, Sohu is designed to run only transformer models, the architecture behind LLMs like ChatGPT.

The company claims Sohu offers dramatically better performance than traditional GPUs while using less energy. This approach could improve AI infrastructure as the industry grapples with increasing power consumption and costs.

Key details:

Etched raised $120 million Series A funding to work on Sohu
Sohu will be manufactured using TSMC 4nm process
The chip can deliver 500,000 tokens per second for Llama 70B
One Sohu server allegedly replaces 160 H100 GPUs
Etched claims Sohu is going to be 10x faster and cheaper than Nvidia's next-gen Blackwell GPUs

Source: https://www.etched.com/announcing-etched

[LoudLemur]

Good model for coding:

https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct

[Kubernetes]

Groq is the AI infrastructure company that delivers fast AI inference.

The LPU Inference Engine by Groq is a hardware and software platform that delivers exceptional compute speed, quality, and energy efficiency.

Groq, headquartered in Silicon Valley, provides cloud and on-prem solutions at scale for AI applications. The LPU and related systems are designed, fabricated, and assembled in North America.

https://groq.com

Since I use this with Llama 3 70B I don't have a need for GPT 3.5 anymore. GPT 4 is too expensive IMHO

[LoudLemur]

@Kubernetes Thanks. How do you actually sign up for Groq, as their stych servers don't seem to be working and they seem to require a Github account for registration

[Kubernetes]

@LoudLemur I did sign up with my Github Account...

[LoudLemur]

Llama 3.1 405b released. Try here: https://www.meta.ai

[LoudLemur]

Jensen Huang, CEO of Nvidia, says we will all become CEOs with AIs reporting to us. At the moment, what do you think would be the best platform/dashboard for unifying your interactions across several AIs and trying to have them work together?