Disk often full, cannot be extended, what remediations exist?

[SansGuidon]

Thanks folks
I've read the guide at https://docs.cloudron.io/storage/#storage and I'm not sure how to proceed then. The guide mentions we can move the data directory but not to a Cloudron volume, so this should be an external volume, is it where Hetzner extra volumes comes into play? I guess then I might be a tutorial away from this first part of the solution.

But these are still dragons to me:

1. The guide advises to not use symlinks but the example commands in the same guide at https://docs.cloudron.io/storage/#default-data-directory
   contain symlink afaik, right?

2. As far as I understand it, the guide seems to cover the case for a full migration of docker images or apps data but it is unclear to me how to tackle this only for specific apps, and if that's a supported scenario. I mean maybe the guide's purpose is to migrate all /app/data volume to an external volume, that could work but then this renders my VPS main partition quite useless, right?

3. I also read several times it was advised not to touch the Cloudron server configuration except via the UI, to ensure future Cloudron upgrades, backups, restore etc would work.
   I guess this is a specific case that requires anyway to touch the terminal, but is it well supported by Cloudron (future upgrades etc)?

I've checked the disk usage to decide what to migrate in my case, I believe migrating the appsdata makes more sense in this case. But the docker images take as much space.

root@ubuntu-cloudron-16gb-nbg1-3:/home/yellowtent# du -hs  appsdata/ boxdata/ platformdata/
28G	appsdata/
49M	boxdata/
4.1G	platformdata/
root@ubuntu-cloudron-16gb-nbg1-3:/home/yellowtent# du -hs /var/lib/docker/
156G	/var/lib/docker/
root@ubuntu-cloudron-16gb-nbg1-3:/home/yellowtent# docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          36        36        27.24GB   2.511GB (9%)
Containers      42        42        0B        0B
Local Volumes   86        86        11.6GB    0B (0%)
Build Cache     0         0         0B        0B

Thanks in advance for the clarifications

[BrutalBirdie]

There is a difference between Cloudron Volumes and Hetzner Volumes.
In the guide is written:

Do not use a Cloudron Volume as storage location for the directories below. Set up fstab or systemd mount manually.

@SansGuidon said in Disk often full, cannot be extended, what remediations exist?:

The guide advises to not use symlinks

It states:

While symlinking various directories will work, this is not supported.

It works, but is not officially supported.
Meaning, yes you can do that, but when you have an issue, you will get less or no support.

@SansGuidon said in Disk often full, cannot be extended, what remediations exist?:

As far as I understand it, the guide seems to cover the case for a full migration of docker images or apps data but it is unclear to me how to tackle this only for specific apps, and if that's a supported scenario.

That what Cloudron Volumes are for.
You can add a Hetzner Volume and mount it to e.g /mnt/hetzner-volume/ and then in the Cloudron Dashboard at volumes do the following:

Then, for each app you can move the data directory.
Like it is described in the doc https://docs.cloudron.io/apps/#data-directory and https://docs.cloudron.io/volumes

@SansGuidon said in Disk often full, cannot be extended, what remediations exist?:

I also read several times it was advised not to touch the Cloudron server configuration except via the UI, to ensure future Cloudron upgrades, backups, restore etc would work.
I guess this is a specific case that requires anyway to touch the terminal, but is it well supported by Cloudron (future upgrades etc)?

Let me quote the doc again:

While symlinking various directories will work, this is not supported.

Officially, this is not supported to reduce support.
But it does work.
And yes, you should not touch the underlaying Server. But it is still your server. If you know what you do and can handle certain issues yourself you can do tasks on the root level.

---

I am doing this for almost all my servers and can tell you, I rarely had issues.
The only issues that ever appeared was when Hetzner Cloud Volumes had some outages (which rarely happens).

[BrutalBirdie]

And here is my terminal output from my personal Cloudron:

root@my-hackradt-com:~# ls -lah /home/yellowtent/
total 32K
drwxr-xr-x 5 yellowtent yellowtent 4.0K Jul  2 18:49 .
drwxr-xr-x 4 root       root       4.0K Oct  3  2024 ..
-rw-r--r-- 1 yellowtent yellowtent  220 Mar 31  2024 .bash_logout
-rw-r--r-- 1 yellowtent yellowtent 3.7K Mar 31  2024 .bashrc
-rw-r--r-- 1 yellowtent yellowtent    0 Jul 16  2024 .cloud-locale-test.skip
drwx------ 2 yellowtent yellowtent 4.0K Nov  9  2024 .gnupg
-rw-r--r-- 1 yellowtent yellowtent  807 Mar 31  2024 .profile
drwx------ 2 yellowtent yellowtent 4.0K Oct  3  2024 .ssh
lrwxrwxrwx 1 root       root         54 Jul  2 18:48 appsdata -> /mnt/my-hackradt-com-Cloudron-Data/yellowtent/appsdata
drwxr-xr-x 8 yellowtent yellowtent 4.0K Jul 21 12:02 box
lrwxrwxrwx 1 root       root         53 Jul  2 18:49 boxdata -> /mnt/my-hackradt-com-Cloudron-Data/yellowtent/boxdata
lrwxrwxrwx 1 root       root         58 Jul  2 18:49 platformdata -> /mnt/my-hackradt-com-Cloudron-Data/yellowtent/platformdata

I did not move the Docker Images Location since I did not need that for this server.
And here is a screenshot of my Hetzner UI.
I've used this setup for 10 months with little to no issues.

[SansGuidon]

Thank you, this clarifies things
Now the pros and cons of opting for either hetzner volumes vs using cloudron volumes is unclear to me, you mention the data directory can also be moved app-per-app via Cloudron UI. I've opted for moving the appsdata folder manually for all apps to a new location, following the guide at https://docs.cloudron.io/storage/#default-data-directory but this seems a different strategy than switching data directory from Cloudron Admin UI and mounting Cloudron volumes.

[BrutalBirdie]

@SansGuidon said in Disk often full, cannot be extended, what remediations exist?:

Now the pros and cons of opting for either hetzner volumes vs using cloudron volumes

You can use a Hetzner Volume with Cloudron Volumes.
What you should not do is use a Hetzner Volume as a Cloudron Volume for a full move which is described in https://docs.cloudron.io/storage/#storage.
Example, you get a 100GB Hetzner Volume, add this in the Cloudron Dashboard as a volume and then move /home/yellowtent/$DIR into this location.
This, you should not do!

---

In my opinion, the pro of using a Cloudron Volume and moving each app to this volume with the GUI is, you stick to the way it is meant to be done and what is officially supported.

On the other hand, I am lazy, I don't want to move every app, I have 30x+ apps.
I just move the whole thing and also include:

lrwxrwxrwx 1 root       root         53 Jul  2 18:49 boxdata -> /mnt/my-hackradt-com-Cloudron-Data/yellowtent/boxdata
lrwxrwxrwx 1 root       root         58 Jul  2 18:49 platformdata -> /mnt/my-hackradt-com-Cloudron-Data/yellowtent/platformdata

This way, all appsdata, all boxdata and all platformdata is on a separate volume.
And when I need more storage, I just increase the Hetzner Volume, resize it in the terminal, and I am done.

[SansGuidon]

Thanks for the advise. Currently I'm in a weird situation, as I've followed the guide step by step and Cloudron starts but my apps do not show any data anymore.

[BrutalBirdie]

I will send you a DM.

[jdaviescoates]

@BrutalBirdie said in Disk often full, cannot be extended, what remediations exist?:

I really had issues.

s/really/rarely

[BrutalBirdie]

@jdaviescoates Good Catch

[SansGuidon]

I solved the problem, not sure what had the most impact:

Change permissions on my mount points/symlinks:

chown yellowtent:yellowtent /mnt/appsdata
chown yellowtent:yellowtent /mnt/appsdata/appsdata

Reboot: not mentioned in the guides but after changing permissions and rebooting, the problem was solved compared to several attempts before this, without reboot and with improper permissions changes.
The permissions and reboot parts are not mentioned in the guide for doing those operations if I'm right, so I'm not sure this was really needed, anyway I prefer to share

Thanks to @BrutalBirdie for all the feedback above and the proposed support (in DM), this community is awesome!