Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Discuss
  3. AI on Cloudron

AI on Cloudron

Scheduled Pinned Locked Moved Discuss
a.i
250 Posts 15 Posters 91.2k Views 18 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • L Offline
    L Offline
    LoudLemur
    wrote on last edited by
    #230

    48GB VRAM from China

    https://wccftech.com/chinese-gpu-manufacturers-push-out-support-for-running-deepseek-ai-models-on-local-systems/

    robiR 1 Reply Last reply
    0
    • L LoudLemur

      48GB VRAM from China

      https://wccftech.com/chinese-gpu-manufacturers-push-out-support-for-running-deepseek-ai-models-on-local-systems/

      robiR Offline
      robiR Offline
      robi
      wrote on last edited by
      #231

      @LoudLemur are these on Alibaba?

      Conscious tech

      L 1 Reply Last reply
      0
      • robiR robi

        @LoudLemur are these on Alibaba?

        L Offline
        L Offline
        LoudLemur
        wrote on last edited by
        #232

        @robi I haven't seen them there. A Chinese ID like a wechat or phone might be needed. I'd be wary of knock offs and consider what people who have bought Chinese steel have had to say.

        1 Reply Last reply
        1
        • L Offline
          L Offline
          LoudLemur
          wrote on last edited by
          #233

          MCP (Model Context Protocol)
          https://modelcontextprotocol.io/introduction

          According to the CEO of Anthropic, this is what is going to be writing 90% of code in 6 months, and almost all of it by the end of the year.

          You can see some uses here:
          https://github.com/punkpeye/awesome-mcp-servers

          The brilliant "Fireship" has a fast introduction to MCP:
          https://odysee.com/@fireship:6/i-gave-claude-root-access-to-my:b

          It would require somebody with something like @robi or @Kubernetes skill levels to be able to use it or make worth of it on Cloudron.

          necrevistonnezrN 1 Reply Last reply
          1
          • robiR Offline
            robiR Offline
            robi
            wrote on last edited by robi
            #234

            Aww, thanks @LoudLemur

            Actually it's already part of the update for OpenWebUI which can use it.

            There are also many MCP directory websites out there who are keeping lists of many as well as some better ones who are letting you run them remotely. One such is Smithery.ai

            Conscious tech

            jdaviescoatesJ 1 Reply Last reply
            1
            • robiR robi

              Aww, thanks @LoudLemur

              Actually it's already part of the update for OpenWebUI which can use it.

              There are also many MCP directory websites out there who are keeping lists of many as well as some better ones who are letting you run them remotely. One such is Smithery.ai

              jdaviescoatesJ Offline
              jdaviescoatesJ Offline
              jdaviescoates
              wrote on last edited by
              #235

              @robi said in AI on Cloudron:

              One such is Smithery.com

              I think you mean https://smithery.ai/

              I use Cloudron with Gandi & Hetzner

              robiR 1 Reply Last reply
              1
              • jdaviescoatesJ jdaviescoates

                @robi said in AI on Cloudron:

                One such is Smithery.com

                I think you mean https://smithery.ai/

                robiR Offline
                robiR Offline
                robi
                wrote on last edited by
                #236

                @jdaviescoates Thanks again for the correction.

                Conscious tech

                1 Reply Last reply
                1
                • L LoudLemur

                  MCP (Model Context Protocol)
                  https://modelcontextprotocol.io/introduction

                  According to the CEO of Anthropic, this is what is going to be writing 90% of code in 6 months, and almost all of it by the end of the year.

                  You can see some uses here:
                  https://github.com/punkpeye/awesome-mcp-servers

                  The brilliant "Fireship" has a fast introduction to MCP:
                  https://odysee.com/@fireship:6/i-gave-claude-root-access-to-my:b

                  It would require somebody with something like @robi or @Kubernetes skill levels to be able to use it or make worth of it on Cloudron.

                  necrevistonnezrN Offline
                  necrevistonnezrN Offline
                  necrevistonnezr
                  wrote on last edited by necrevistonnezr
                  #237

                  @LoudLemur said in AI on Cloudron:

                  MCP (Model Context Protocol)
                  https://modelcontextprotocol.io/introduction

                  According to the CEO of Anthropic, this is what is going to be writing 90% of code in 6 months, and almost all of it by the end of the year.

                  MCP has been adopted by OpenAI and will hence be the de-facto standard https://techcrunch.com/2025/03/26/openai-adopts-rival-anthropics-standard-for-connecting-ai-models-to-data/

                  In a post on X on Wednesday, OpenAI CEO Sam Altman said that OpenAI will add support for Anthropicโ€™s Model Context Protocol, or MCP, across its products, including the desktop app for ChatGPT. MCP is an open source standard that helps AI models produce better, more relevant responses to certain queries.

                  1 Reply Last reply
                  2
                  • robiR Offline
                    robiR Offline
                    robi
                    wrote on last edited by robi
                    #238

                    It would be great to have additional MCP support in OWUI.
                    @girish or @nebulon can you take a look?

                    It requires a few things added to the package it seems.

                    https://github.com/open-webui/openapi-servers

                    Also, the import functions feature seems to be broken, although one can use the +
                    Add function manually and copy/paste the code from https://openwebui.com/functions

                    Conscious tech

                    1 Reply Last reply
                    3
                    • robiR Offline
                      robiR Offline
                      robi
                      wrote on last edited by robi
                      #239

                      For those that want to experience how simple it is to build an agent and imagine implementing it on Cloudron.. read on: https://github.com/voxmenthe/coding-agent/blob/main/tutorial.md

                      Conscious tech

                      1 Reply Last reply
                      2
                      • L Offline
                        L Offline
                        LoudLemur
                        wrote on last edited by
                        #240

                        RAG with documents in a Docker
                        https://github.com/SciPhi-AI/R2R

                        1 Reply Last reply
                        0
                        • L Offline
                          L Offline
                          LoudLemur
                          wrote on last edited by
                          #241

                          StabilityMatrix

                          If you use a lot of inferencing software, one thing this can do is have the models kept in one folder, which saves storage and bandwidth.

                          https://github.com/LykosAI/StabilityMatrix/releases/tag/v2.14.0

                          1 Reply Last reply
                          0
                          • L Offline
                            L Offline
                            LoudLemur
                            wrote on last edited by
                            #242

                            AI Search with Lepton
                            This helps you have an ai language model which makes internet searches.

                            An interesting feature that Cloudron might like is its one-click deployment tool.

                            https://github.com/leptonai/search_with_lepton?tab=readme-ov-file

                            1 Reply Last reply
                            0
                            • jdaviescoatesJ Offline
                              jdaviescoatesJ Offline
                              jdaviescoates
                              wrote on last edited by
                              #243

                              Apparently this is one of the best open source models yet:

                              https://qwenlm.github.io/blog/qwen3/

                              I use Cloudron with Gandi & Hetzner

                              L 1 Reply Last reply
                              2
                              • L Offline
                                L Offline
                                LoudLemur
                                wrote on last edited by
                                #244

                                LLM VRAM Calculator

                                If you would like to know what models your VRAM can handle, you can check with this online tool:

                                https://apxml.com/tools/vram-calculator

                                1 Reply Last reply
                                2
                                • jdaviescoatesJ jdaviescoates

                                  Apparently this is one of the best open source models yet:

                                  https://qwenlm.github.io/blog/qwen3/

                                  L Offline
                                  L Offline
                                  LoudLemur
                                  wrote on last edited by
                                  #245

                                  @jdaviescoates I just tried it.
                                  Qwen 3 is very good!

                                  1 Reply Last reply
                                  1
                                  • L Offline
                                    L Offline
                                    LoudLemur
                                    wrote last edited by
                                    #246

                                    Live Search is available on Grok's API without charge for a limited time for beta testing:

                                    https://docs.x.ai/docs/guides/live-search

                                    search through realtime data from ๐•, the internet, trending news, RSS feeds,

                                    necrevistonnezrN 1 Reply Last reply
                                    3
                                    • L LoudLemur

                                      Live Search is available on Grok's API without charge for a limited time for beta testing:

                                      https://docs.x.ai/docs/guides/live-search

                                      search through realtime data from ๐•, the internet, trending news, RSS feeds,

                                      necrevistonnezrN Offline
                                      necrevistonnezrN Offline
                                      necrevistonnezr
                                      wrote last edited by necrevistonnezr
                                      #247

                                      @LoudLemur said in AI on Cloudron:

                                      Live Search is available on Grok's API without charge for a limited time for beta testing:

                                      https://docs.x.ai/docs/guides/live-search

                                      search through realtime data from ๐•, the internet, trending news, RSS feeds,

                                      In the US, Musk is simply walking into the authorities with a presidential power of attorney and copying everything he can. Once the files are copied, they end up in the system of Musk's X AI. Or they are poured into Palantir software with the aim of creating deportation lists.
                                      So please boycott this PoS and his PoS endeavors.

                                      marcusquinnM 1 Reply Last reply
                                      0
                                      • necrevistonnezrN necrevistonnezr

                                        @LoudLemur said in AI on Cloudron:

                                        Live Search is available on Grok's API without charge for a limited time for beta testing:

                                        https://docs.x.ai/docs/guides/live-search

                                        search through realtime data from ๐•, the internet, trending news, RSS feeds,

                                        In the US, Musk is simply walking into the authorities with a presidential power of attorney and copying everything he can. Once the files are copied, they end up in the system of Musk's X AI. Or they are poured into Palantir software with the aim of creating deportation lists.
                                        So please boycott this PoS and his PoS endeavors.

                                        marcusquinnM Offline
                                        marcusquinnM Offline
                                        marcusquinn
                                        wrote last edited by
                                        #248

                                        @necrevistonnezr Interesting take. Companies are already more powerful that governments in many ways.

                                        Web Design https://www.evergreen.je
                                        Development https://brandlight.org
                                        Life https://marcusquinn.com

                                        necrevistonnezrN 1 Reply Last reply
                                        0
                                        • marcusquinnM marcusquinn

                                          @necrevistonnezr Interesting take. Companies are already more powerful that governments in many ways.

                                          necrevistonnezrN Offline
                                          necrevistonnezrN Offline
                                          necrevistonnezr
                                          wrote last edited by
                                          #249

                                          @marcusquinn itโ€™s from an interview with one of the best-known network political activists in Germany Markus Beckedahl. The trained IT merchant created the online medium netzpolitik.org and is one of the founders of the re:publica conference. He is also co-founder of the Society for Freedom Rights, who bring lawsuits dealing with constitutional rigths to court on behalf of citizens. He blogs at digitalpolitik.de.
                                          See https://www.heise.de/news/Interview-mit-Markus-Beckedahl-Musk-putscht-mit-Software-statt-Panzern-10379742.html (German)

                                          marcusquinnM 1 Reply Last reply
                                          1
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Bookmarks
                                          • Search