Understanding external storage

[Emilien-Lambert]

Hello,

I've been testing Cloudron for a few days and have some questions. Recently, I tried connecting an external hard drive via a Raspberry to my Cloudron server using NFS and VPN, but it didn't work as expected link to the topic

My main question is:
How do you manage large data storage (1-3 TB) on a VPS server, considering storage is expensive and limited (40 GB in my case)? I had thought of using a Raspberry to easily connect 5 TB, but that seems unfeasible. What are my options for increasing storage at a reasonable cost?

Thanks

[jdaviescoates]

@Emilien-Lambert I think it kinda depends on what sort of data and apps you're talking about.

e.g. for Navidrome I've got a 1TB Hetzner Storage Box mounted as a Cloudron Volume with all my music files.

For all my Fediverse apps (Peertube, Mastodon, Pixelfed) I've connected them (via config in their env files, by following upstream docs) to S3 Object Storage at Scaleway.

[Emilien-Lambert]

@jdaviescoates, thanks for your reply. I'm running into a problem: I can't figure out how to integrate object storage into an application. I know it's possible to connect, for example, an S3 for backups with Backblaze and authentication tokens, but this seems to be limited to backups.

Following the discussions, I understood that the OpenVPN application doesn't offer the possibility of connecting a remote device to access a shared folder. So, if I install JellyFin, I wonder how to access my movies. The storage options in the application settings seem only linked to local server volumes.

I haven't found any tutorials or instructions on how to connect storage.

[Emilien-Lambert]

By the way, I don't understand what the openVPN application is for either, if I can't connect a computer in my house to the cloudron server to share data. Would anyone have a use case for me to understand please?

[jdaviescoates]

@Emilien-Lambert said in Understanding external storage:

I can't figure out how to integrate object storage into an application.

Which application?

@Emilien-Lambert said in Understanding external storage:

So, if I install JellyFin, I wonder how to access my movies.

Access your movies where?

I think what you want to do is

1. install Jellyfin using Cloudron installed on a VPS somewhere, then
2. enable that to access movies stored on a machine in your house

Is that right?

I've no idea how to do that, but I think it might technically be possible using the Dynamic DNS stuff https://docs.cloudron.io/networking/#dynamic-dns

What I'd do (what I've done with my music) is upload (using rsync) all your movies to a Hetzner Storage Box, then mount that as a Clourdon Volume to your Jellyfin app and edit the Jellyfin config to tell it to use that mount.

e.g. I've got this Volume set-up which mounts a Hetzner Storage Box:

And then in the Navidrome app Storage settings I've mounted that Volume to the app:

And then I've used the File Manager to edit the Navidrome app config file to tell it to use the location of that Volume:

Presumably it'd be possible to do something very similar with Jellyfin.

@Emilien-Lambert said in Understanding external storage:

By the way, I don't understand what the openVPN application is for either, if I can't connect a computer in my house to the cloudron server to share data. Would anyone have a use case for me to understand please?

I don't really know much about this either, but I once had a little play around trying to get access to a nice documentary which is currently only available on Netflix in Germany. I'm in the UK, so I can't access it. By I thought "hey server is in Germany, so maybe I could use OpenVPN to connect via that"... so I installed OpenVPN on my server, and then connect to it from my laptop and then tried to access German Netflix... and I think it likely would've worked - except I'd forgotten my server is actually in Finland! (I've just realised I do actually have another small VPS that is in Germany so I may go and try to do that again to see if it does work...)

Edit: oh, actually I had try to do it using my German VPS server before anyway. Not sure why it didn't work (I guess Netflix is too clever)

[girish]

@Emilien-Lambert said in Understanding external storage:

How do you manage large data storage (1-3 TB) on a VPS server, considering storage is expensive and limited (40 GB in my case)?

Option 1: If you large amounts of data at home, is it an option to run Cloudron in a home server? Note that you can have valid certs, publicly accessible websites, working mobile apps etc when hosting at home. All you need is a public IP (even dynamic will do).

Option 2: purchase a hetzner storage box which has one of the best prices on the market - https://www.hetzner.com/storage/storage-box . You can mount these as a Cloudron volume and then apps can access them. Note that they have storage boxes in couple of countries, so choose the one closest to your server. Ideally, purchase the server from hetzner as well. This will involve copying your stuff from home to storage box (a bit painful but doable).

Option 3: what you are trying now I think. Instead of setting up a VPN tunnel to make NFS work, I recommend using a SSH tunnel. See https://stackoverflow.com/questions/5280827/can-someone-explain-ssh-tunnel-in-a-simple-way and https://superuser.com/questions/1540212/how-to-provide-nfs-access-over-an-ssh-tunnel for ideas. Basically, the rpi would create a ssh reverse tunnel for the NFS port 2049 to Cloudron server . Then, on Cloudron, you can just setup a volume that does a NFS connection to the localhost mapped port.

[girish]

Also, the goal of OpenVPN app is to tunnel traffic via Cloudron. So, you can purchase a server in say UK, setup Cloudron+OpenVPN app there. Your devices can then connect the OpenVPN server to tunnel traffic via UK. I have a server in India with a VPN and I use this for accessing indian sites/content. Many sites are geoblocked

[Emilien-Lambert]

@girish and @jdaviescoates

Thank you very much for all these precious explanations I understand much better and the different options you have presented to me should do the trick. Thank you for taking the time to be an educator