Cloudron Forum

@james In our case (we're have some old enterprise Nvidia GPUs), we found that installing the Nvidia drivers and CUDA toolkit is ultimately not difficult once you know what to do. Our problem with making that useful on Cloudron (per this old thread) was that we couldn't start Cloudron's OpenWebUI package with GPU support. So even though we had our GPU hardware working on a Cloudron server, we could never use it with a Cloudron instance of OpenWebUI/Ollama.

I suspect the process of installing the Nvidia drivers and CUDA toolkit is generic enough that it could be supported by Cloudron somehow. However we don't have any GPUs from other enterprise brands (e.g. AMD, Intel) so we never tried that, and I imagine supporting drivers for consumer GPUs is an impossible maze of complexity.

We would LOVE it if you could support the Nvidia GPU use case as a basic option.

Firstly, this is wonderful news. I see that your Ollama package appears to include an authentication capability, which is something that vanilla Ollama is missing. When coupled with Cloudron's app management features, that makes it extremely helpful.

However... Without GPU support or crazy fast CPUs and a big load of fast RAM, isn't Ollama self hosting almost useless?

Note - Here's an old thread where we discussed some related topics.

@marcusquinn n8n is very much not open source, even the "free" version has significant restrictions on how you can use it ("internal" business use - which is not well defined - or non-commercial use only). If you're self hosting and using it for commercial purposes, you should consult the n8n licensing team about purchasing a (very, very, very expensive in my opinion) licence.

Thanks @joseph - I will try that!

Meanwhile, a small update: The disk filling process appears to have magically stopped. I don't know when. So our Rocket.Chat appears to be back to normal operation. I don't see any obvious clues in the package updates, although there are updates to the apps engine, maybe it was that. Our instance is backing up and updating happily again.

It wasn't me!

Good question @joseph - Unfortunately I can't see the settings in Rocket.Chat right now, my settings screen is blank. (I double checked and I do have admin user access.) So I assume it is whatever is default. A suggestion above is to change the logs storage and see what happens. I will give that a go when I get some time to work out how to alter the invisible settings.

Meanwhile, I have observed that while the disk filling process does not kick off straight away, it always kicks off when the backup process starts. Taking a backup seems to trigger the process with 100% reliability. (In fact this might be why the disk fills up overnight, as I had auto backups enabled.) Then the disk fills and the backup fails. So I have no app level backups for Rocket.Chat at the moment.

Regarding the separate problem of blank settings, I found that the Admin group had checkboxes DISABLED for all the settings configurations. I don't know what caused that or whether it's important, we've never touched this screen. I have switched these on but my settings are still blank so far.

Quick update: Upgrade to package 2.55.0 with Rocket.Chat server 7.3.0 did not fix the problem (though I didn't think it would based on the RC release notes).

Attempts to get help on the Rocket.Chat community forum appear to have failed because I encountered the rudest and most self centred and unhelpful "Community Liaison Officer" I've ever had the misfortune to meet, a genuinely toxic individual. However, my glass is half full: the silver lining is that it reinforced my sincere appreciation of the community in this forum and the Cloudron team.

This would certainly be a very valuable addition to Cloudron. I have upvoted. That said, I guess that the configuration process could be challenging since there doesn't appear to be any point-and-click UI for that.

I forgot an important advantage: You're supporting open source.

Sorry, I didn't answer your original question directly...

Real world server specs for OpenWebUI itself are very low. My Cloudron OpenWebUI app instance fits into a few Gb of storage, barely uses any CPU on its own, and runs in well under 1 Gb of RAM.

But if you want to use the embedded ollama system to interact with locally hosted LLMs, your server needs to support the actual LLMs aswell as OpenWebUI. So you need all of this:

• Enough disk storage for all the models you want to use.
  • Individual models you can typically run locally for a reasonable cost range from 2-3 Gb (e.g. for a 3B model) up to 40-50Gb (e.g. for a 70B model). You might want to store multiple models.
• Enough RAM (or VRAM) to fully load the model you want to use into memory, separately for each concurrent chat.
  • To roughly calculate, you need the size of the model file plus some room for chat context depending on how much you want it to know/remember during chats, e.g. 3-6Gb per chat for 3-8B models, more for the bigger ones.
• Enough CPU (or GPU) compute power to run the model fast.
  • For tiny (3-8B) models, expect 1-2 minutes per chat response on a typical CPU+RAM system and don't imagine you can use bigger models at all, or seconds per chat response using GPU+VRAM. (Note: You might do better than that on the very latest CPUs, but GPU+VRAM is still going to be hundreds of times faster.)
• If you're using CPU+RAM (as opposed to GPU+VRAM):
  • You'll find that your disk I/O will be hammered (particularly during model loading) too, so you'll want very fast SSDs.
  • Expect your CPU and your RAM to be fully consumed during inference (chats), so don't expect to be running other apps on your server at the same time.

In short, I'm not sure that a VPS hosted OpenWebUI instance running only on CPU+RAM is ever going to be useful for self hosted LLMs.

Unfortunately, even if you have a GPU on your virtual server, even if you get under the hood and install your GPU drivers on the Ubuntu operating system, currently Cloudron's OpenWebUI app installation won't use your GPU. So on Cloudron you're stuck with CPU+RAM.

But that is not as gloomy as it sounds... To answer your next question...

@timconsidine said in Real-world minimum server specs for OpenWebUI:

Using “out the box” with local default model.
Is there any point to the app to use with publicly hosted model ?

Yes, there is a point. Your use cases for handling data privately are more limited, certainly, but there are some outstanding advantages to doing this, particularly on Cloudron.

• You're storing your data (including chats and RAG data) on a system you control.
  • Although you're still sending your chats and data within them to the public model, you at least control what you can do with the storage of your chats and data.
• You can download, backup, and always access your chats, or move them to a different OpenWebUI server, even if your connection to the public model is severed.
• You can interact with multiple public and private models via a single interface, even within each chat. None of the public platforms let you talk to the others.
  • E.g. OpenWebUI has some pretty cool features to let you split chat threads among different models, and let models "compete" with each other using "arena" chats. We've found this to be invaluable in our business because a lot of optimizing AI usage is about experimentation and finding the best tool for the task at hand.
• You can install and manage your own prompt libraries, system prompts, workspaces (like "GPTs" in ChatGPT), coded tools and functions (OpenWebUI has some cool integrated Python coding capabilities in this area), in a standard way across every LLM that you interact with, and without storing your code and extended data in the public cloud.
• You can brand your chat UI according to your company or client, and modify/integrate it in other ways. OpenWebUI is flexible and open source.
• You can centrally connect to other apps that you self-host for various workloads including data access and agent/workflow automation without needing to upload and manage all that stuff in public systems.
  • E.g. some apps running on Cloudron that can give your AI interactions super powers include:
    • N8N for workflow automation
    • Nextcloud for data storage and management
    • Chat and notification apps
    • BI apps like Baserow and Grafana
• You can manage and segregate branded multi-user access to different chats and different AIs, either in a single OpenWebUI instance, or (since app management on Cloudron is so bloody easy), different instances on different URLs.
• In the future when you switch to self hosted LLMs or other integrations, there's little no migration. You just switch off the public API connectors and redirect them to your own models and tools, because you managed your data and chats and code integrations locally from the outset.
• And more. I'm sure I didn't think of everything.

By the way, plenty of these advantages are either because of or enhanced by running on Cloudron. Cloudron is great.

I haven’t tried DeepSeek locally but might be worth a shot for privacy. I wouldn’t use it otherwise.

I agree with that decision wholeheartedly. Well, unless you're talking with DeepSeek about stuff that you want the whole world to know and learn from. Then, go nuts.

@timconsidine Are you trying to use locally hosted ollama models, or have you wired up API keys for the public cloud models like ChatGPT or DeepSeek(*) in your OpenWebUI instance?

If you're experiencing unusable slowness for locally hosted models, it might be because OpenWebUI on Cloudron out of the box is running with CPU+RAM only (not GPU+VRAM). Even for tiny models, that's going to be very slow even with very fast CPUs.

I'd be surprised if you're finding OpenWebUI to be slow with the public cloud models. There will be some latency through API calls between your Cloudron server and the online model. but I'd be surprised if you didn't find it to be nearly as fast as using the online hosted versions directly.

(*) By the way, if you're using DeepSeek online and not self-hosted, please assume every interaction is being read at the other end. There are no privacy controls. And even with ChatGPT and the others, I'd suggest reading the terms and conditions of your API usage carefully and considering which jurisdiction you're sending your data and chats to.

Quick update: We now have Rocket.Chat on the 2.54.3 package (ran a manual update without a backup), i.e. the small update which sets the Deno cache directory. It has not fixed the problem. Nothing useful from the Rocket.Chat forums yet. So still working on it.

@stoccafisso I can confirm that for me this is Rocket.Chat flakiness, not Cloudron. I fear your restoration process won't help if Rocket.Chat is just going to kick off its log filling process again.

@girish I haven't yet tried moving logs to the filesystem, because (very strangely) my Rocket.Chat settings page currently appears to be empty, just a search box:

That's a Rocket.Chat mystery I haven't figured out yet.

I did try the scheduled mongodb log clearing idea directly in the app container, but it didn't seem to help. Because I can't see what's actually happening in the database, I don't know if that's because the data is filling up faster than the query runs, or because of database locking because the database is so big (100s of Gb), or something to do with virtual container file system strangeness, or something else.

At this stage my server seems to have fallen into this pattern:

• Rocket.Chat fills the disk
• Rocket.Chat attempts an update (I have auto updates on)
• Update fails because the backup fails (no disk space to prepare the backup, even though my backups go offsite to Backblaze)
• Rocket.Chat restarts, at which point the "filled" disk space moves from within the Rocket.Chat container to the virtual file system (reported as "everything else" on the Cloudron System Info page)
• A reboot releases the disk space from the virtual file system
• Start over...

The disk filling doesn't seem to start immediately any more. I don't know what triggers it.

So that means with a reboot every day or so, the server is more or less operational for general usage, except for a while right at the end when the disk is full.

I'm hoping a Rocket.Chat update will arrive soon that makes this go away.

Oh, the problem is not gone.

I'm not sure when that happened, Rocket.Chat had stopped working again this morning when I logged in so that's when I checked. So I assume it's Rocket.Chat again. Since the app wasn't working the logs are full of errors, so I couldn't easily see anything useful there. Since the Rocket.Chat container's file system changed in the last update, I can't seem to see the .bson files any more so I can't confirm that was the problem and I'm not sure what else to check.

There were some other automatic app updates over the last two nights (though not Rocket.Chat because it's up to date), so I suppose it's possibly not Rocket.Chat this time, I'm just assuming.

I can't see that data anywhere on the Cloudron disk from the server console. A du -h --max-depth=1 at the root level on the server shows Cloudron is using in expected the amount of space taken up by the apps in the coloured parts of the bar above even though df says the disk is full. The 'everything else' data doesn't seem to exist. So I can't tell what the data is. It appears to be living in the overlay filesystem until a Cloudron reboot, at which time it disappears. (Sorry I didn't take a screenshot.) I know nothing of these virtual file systems, so apologies if I'm explaining that wrong, just reporting what I can see.

So after a reboot it's all good again. After 30 minutes or so, there's no evidence so far that the disk is filling up or anything is changing on the disk yet:

Um... The problem may have magically disappeared.

Yep, the disk filled up. The update failed. The app restarted. At which point, storage snapped back to its original state - 300Gb gone in the blink of an eye. I started the 2.54.2 package update again, skipping the backup just in case the snapshot process hung like earlier. It ran. Rocket.Chat is now on the latest version. The disk filling process does not appear to have re-commenced.

I've seen this before. Don't watch Highlander II, it was a horrible update that should never have been made. Just go directly to Highlander III.

Geez... So starting the 2.54.1 package update (which I note upgrades to Rocket.Chat server 7.2.1) appeared to remove the rocketchat_apps_logs.bson file/collection and start over - which didn't happen on the previous update. (I wonder if this has something to do with the server upgrade?) However now I am watching that file grow rapidly during the update process and the update appears to be stuck on this step:

I guess this is going to fill up the disk and then fail. I'm not quite sure what to do next.

(By the way, I did try running the mongo data clearing process offered by @martinv earlier. However it went into a black hole, which I suppose is not surprising with a 200 Gb collection file, and meanwhile the disk was filling up rapidly so I stopped it. I suppose since that was running inside the container, everything zapped back to its original state when I restarted the app. Was it wrong to run that in the app console, should I access the database from outside instead?)

If the disk does fill up and the update stops, and if Rocket.Chat is still running at that point, I'll try the mongo data clearing process again. But I suspect the app won't be running at that point so I won't be able to do anything with the database.

This is not my favourite Rocket.Chat update.

Ok, so that report that "App was updated to {previousVersion}" means the update didn't work, is that right? I'll know that from now on.

I was able to get Rocket.Chat updated to the latest 2.54.1 package. I had to skip the backup process, as the snapshot was not going to finish before the disk filled up. Unfortunately it has not fixed the problem. Disk space started growing right away.

It did make these errors go away though, which is what was happening with 2.54.0:

I note the rocketchat_apps_logs.bson collection did not get updated during this update. This file is now 201Gb.

Oh... Does a local snapshot get taken before it's shipped off to Backblaze? That would make sense and explain the previous backup errors during the app update then; there wasn't enough local disk space.

Also I was wrong about rocketchat_apps_logs.bson not growing. It was (it's up to 158Gb again in under an hour), but perhaps I wasn't seeing the file size update in my remote console while the container was running? This is what's still filling up disk space.

Firstly, thank you @girish and @martinv for being far more useful than a compass at the South Pole! Still nothing useful from the Rocket.Chat forum.

I have set up that cron task to run every 15 minutes, since our log file seemed to be growing very quickly. (After freeing up 40 Gb of disk space yesterday from other apps, our Cloudron server filled up again overnight.) However the cron task seems to be a workaround rather than a solution...

Also in very strange news, the rocketchat_apps_logs.bson file does not seem to be growing any more, even between cron task runs. I'm still not sure what's in that data, I haven't been able to look at it yet, but I'm guessing it's log data for third party Rocket.Chat apps.

Something else is now filling up the disk very quickly, a Gb every few minutes. I haven't figured out what yet. It does seem to correspond to running Rocket.Chat.

I noticed some other strange things and potentially useful notes while working through it:

Firstly, the disk space problem must be very new or we would've seen it ages ago.

I see there was an auto update to the 2.53.0 package last week.

Then the next day, an update to 2.54.0... Or was there?! (I think there was.)

I'm now seeing this behavior on each update, and it wasn't happening before. 2.54.1 landed a few days ago when the same backup error and strange event log effect. I think this is when the problem most likely started.

I still see the Update button after the update and clicking it offers to upgrade to the same version again. I notice this is not the latest package (which is 2.54.2.)

I noticed the 2.54.x series of updates do contain updates to the apps management system. Could it be related?

I noticed that when I manually ran the update, that's when the rocketchat_apps_logs.bson file seemed to diminish in size: it was no longer hundreds of Gb after the update. But I also noticed this:

Strangely, I couldn't find that data anywhere on the file system by digging through du reports; the disk usage across all directories did not add up to the disk usage I could see with df.

After a reboot it seemed to disappear. Is this something strange about the overlay file system that I don't understand?

Something that may or may not be relevant: Our app backups are going to a Backblaze data store, we don't keep them locally.

I'm going to try re-running the app update again to see if I can get to 2.54.2, and to see if the same thing happens with our rapidly growing data disappearing as part of the update.

Hi all,

This is a Rocket.Chat question rather than a Cloudron question, so apologies for coming here, but the Rocket.Chat forum seems to be about as useful as a chocolate teapot.

Our rocketchat_apps_logs.bson file within our docker container is huge: ~266Gb. So it's hogging a lot of useful space on our Cloudron server. I've no idea what's stored in here, but it feels like it might not be critical data. Does anyone know if there's a way to reduce or truncate it, or if it's safe to remove?

Cheers,
Rob

@Lanhild said in ETA for GPU support? Could we contribute to help it along?:

A lot of companies that might deploy Cloudron for its ease of life features don't necessarily have a VPS with a GPU.

Also, (might help you to deepen your Cloudron knowledge) Cloudron packages usually are only one component/application.

Moreover, OpenWebUI is "just" a UI that supports connections to Ollama and isn't affiliated with it. Meaning that Ollama isn't a dependency of it at all.

Excellent points @Lanhild - you've convinced me.

And there are benefits on the Ollama side too. I would appreciate the benefit in using Cloudron to keep our Ollama installation automatically up to date on its own, for instance.

In fact, given our remaining inability to modify the existing Cloudron OpenWebUI app to run with our GPUs, for our small clients we are now thinking this way - I.e. using Cloudron just for the OpenWebUI component and letting them connect to our separately hosted Ollama. It's a bit less convenient than we were hoping, but at least we'll still have segregated data and user management for each client in OpenWebUI.

So now, I also want a Cloudron OpenWebUI app that does not come with bundled Ollama, so that I can be sure these customers don't hammer our CPUs and get frustrated by a slow user experiences.