Accessing mounted volumes in jupyter notebooks

[nigeil]

I have been attempting to provide access to a cloudron volume to users working with jupyterhub.

I have been able to mount the volume to the jupyterhub parent container through the cloudron web interface, and can access it from the web debugging terminal under /media/ as expected.

However, the volume is not accessible in the spawned jupyterlab instances, either under /media/ or /home/jovyan/ (the default home directory in spawned jupyterlab containers).

Digging through some of the code that is generated at run-time to spawn jupyterlab containers (such as /run/jupyterhub/config), I do not see any mention of mounting the directories under /media/ to the child containers.

I imagine the parent container (the jupyterhub server that cloudron directly manages) will need to mount the shared volume to the jupyterlab containers it spawns. However I am not sure how to accomplish this.

I appreciate any thoughts/guidance cloudron users and staff might have - thank you!

[balticpenguin]

Is there already a solution here?

[nebulon]

Currently child container will not inherit the app mounts and thus there is no way to have access to the data in those. We have to revisit how we create subcontainers then and change the behavior of the docker addon then https://docs.cloudron.io/packaging/addons/#docker

[robi]

@nebulon sysbox would help with that.

[balticpenguin]

In the settings of Jupyterhub there are
c.DockerSpawner.volumes and
c.SwarmSpawner.read_only_volumes
(https://jupyterhub-dockerspawner.readthedocs.io/en/latest/api/index.html)

Would this perhaps be a possibility? Currently, however, these config settings do not work.

Since I have day files with data that I need to read and edit. Therefore it is very inconvenient if I would have to upload them manually. Besides, my colleagues also work with the data. So it would be great if there was a directory "shared" which is in the home directory. Great would be a setting in custumconfig.py like
c.DockerSpawner.volumes = {'/media/shareddata' : '/home/jovyan/shared'}

see also https://jupyterhub-dockerspawner.readthedocs.io/en/latest/data-persistence.html#volume-mapping

[girish]

@balticpenguin @nigeil I implemented (cloudron) volume mounting for child containers now in https://git.cloudron.io/cloudron/box/-/commit/740c0fe318351ee48dcb069f43e939daa34538e1 but actually I don't think you need this at all.

I think all you need to do is:

• Create /app/data/shared directory in JupyterHub's file manager.
• Then, in customconfig.py

 c.DockerSpawner.volumes['/app/data/shared'] = '/shared'

• That should be it

Can you please try above? You can then open filemanager. It's important to not "assign" to volumes property but to add to the Dict . The home folder /home/joyvan is already mapped by the package's base config.

[balticpenguin]

@girish I have tested and it works. I can share data with other users. But it is possible to get access to mounted volume from the user container? The mounted volume is only accessible in the Jupyterhub container, but not in the user container.

[girish]

@balticpenguin thanks for testing. I have fixed the volume sharing for user containers with https://git.cloudron.io/cloudron/box/-/commit/740c0fe318351ee48dcb069f43e939daa34538e1 . One thing is when you do volume sharing, the shared volume is not backed up (because Cloudron does not support backing up volumes yet). Is this OK?

[balticpenguin]

That's okay with me.
Unfortunately, I've noticed another problem: I have also included the volume in the Nextcloud app. Files are uploaded here, which I then have to edit in Jupyterhub. But I don't have write permissions for this file. The user of the file is www-data. When I edit or create files in the Juyperhub app, it belongs to the user wcd.

Is there a way to fix this problem?

[girish]

@balticpenguin try https://docs.cloudron.io/volumes/#sharing . You can use setfacl to automatically have new files owned by media group.

[balticpenguin]

If you want to share a volume with Jupyterlab, this is unfortunately not enough. Each user gets their own Jupyterlab container from a notebook image. By default, this is jupyter/datascience-notebook. However, no media group is created in this container. There is also no user cloudron.
Since I created my own image, I could simply create the group media and add the user Jovyan. Now it works.