Cloudron Data Paths - Best Practices

[crazybrad]

@girish Thank you for explaining /var/lib/docker and best practice.

I am a bit confused about your documentation reference to volumes vs. https://docs.cloudron.io/storage/#default-data-directory. I am thinking about moving all data directories to an external ext4 volume that Hetzer has automatically mounted at /mnt/<volname>. Will that work? I also want to make sure that the default Cloudron backup processes will apply. Do I need to move the default location of the Hetzner volume?

[girish]

@crazybrad the approach in the link to symlink all the data directories to external directories will work fine as well. It will backup properly as well.

[michaelpope]

Hey @crazybrad In my experience, it works (including backup), although it does create issues if for whatever reason the volume doesn't mount correctly on a server restart (happens occasionally).

[bmann]

CIFS and NFS look like the only "remote" file systems. Are either of them performant? Are there any hosted options for those?

I guess Volume storage that DO, Hetzner and others offer is the main use case aside from "local" Cloudron installs with local disk?

[crazybrad]

@girish Thank you for clarifying. @michaelpope That's another reason why no one likes to restart their servers:) Appreciate the "heads-up".

[michaelpope]

@bmann CIFS, NFS, are both remote file systems, yeah. SSHFS is a remote file system built on SSH, so that technically counts too. But all of those have somewhat limited functionality.

From what I've seen, most people who use the Mount backup are using it for where your VPS provider gives you a volume to work with. Sometimes they are technically remote too, on the backend, but they act like they are local (I know BUYVM's volumes are based on an RDMA which is basically like direct disk access over ethernet, which is kinda cool - Hetzner probably uses some other technology for their volumes).

[chetbaker]

As expected, volume-based disk storage is 6 times slower than local disk.

Now that the topic is set as "best practices" ( ) I wonder if there's any nice practice some of you do for setting up external Volumes for data (not backups). I'm interested in this considering the size of my Nextcloud instance that gets this giant backup I don't necessarily need.

[bmann]

@michaelpope yeah I was skipping SSHFS as kind of being a temporary hack, whereas CIFS / NFS are designed as remote file systems (total assumption on my part).

I'm thinking about this for scaling storage beyond one machine (not for backup). Nextcloud, Peertube, Pixelfed, and some of the photo management apps are ones that could really eat storage.

[jdaviescoates]

@bmann said in Cloudron Data Paths - Best Practices:

Peertube, Pixelfed

For those (and Mastodon too) I'd strongly recommend connecting them to an S3 compatible object storage and storing everything there instead.

I use Scaleway Object Storage buckets for that myself.

[bmann]

@jdaviescoates OK, but that puts us into custom configuring those apps. Do those env config changes persist when they are Cloudron managed?

I see that being documented on the Mastodon on Cloudron page but it's not on the Peertube or Pixelfed pages.

Between app data, Volumes, and Backups, there are three places to put config around storage. I'd love to see those items be configurable at the Cloudron admin layer.

The wording doesn't totally match, but having Volumes actually be "Storage" or "Volumes and Backup" and having multiple configs there might be a useful evolution.

[jdaviescoates]

@bmann said in Cloudron Data Paths - Best Practices:

@jdaviescoates OK, but that puts us into custom configuring those apps. Do those env config changes persist when they are Cloudron managed?

Yes. Seems to working fine.

@bmann said in Cloudron Data Paths - Best Practices:

I see that being documented on the Mastodon on Cloudron page but it's not on the Peertube or Pixelfed pages.

Yeah, could really do with lots more documentation on how to do it. It's mostly documented upstream, but it is way harder than one might expect and seems to require lots of trial and error. Doesn't help that it seems to be a bit different depending on which S3 provider you're using.

[michaelpope]

@bmann Yeah, in that case, I'd say just use a provider that allows you to buy extra storage inexpensively and mount it as a volume. BuyVM is the one that comes to mind honestly.

You can also use CIFS or NFS, but I don't know how well I'd trust them as primary storage for an app.

Also, as said, some apps support external S3 (although that limits your capability to fully encapsulate your backups).