baserow error on file column upload

[midm]

Hello,

On a fresh install of Cloudron, I'm unable to upload files to Baserow, with followings error in logs :

Dec 28 00:08:13 ERROR 2022-12-27 23:08:13,289 django.request.log_response:224- Internal Server Error: /api/user-files/upload-file/
Dec 28 00:08:13 Traceback (most recent call last):
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/asgiref/sync.py", line 482, in thread_handler
Dec 28 00:08:13 raise exc_info[1]
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/django/core/handlers/exception.py", line 38, in inner
Dec 28 00:08:13 response = await get_response(request)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/django/core/handlers/base.py", line 233, in _get_response_async
Dec 28 00:08:13 response = await wrapped_callback(request, *callback_args, **callback_kwargs)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/asgiref/sync.py", line 444, in __call__
Dec 28 00:08:13 ret = await asyncio.wait_for(future, timeout=None)
Dec 28 00:08:13 File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
Dec 28 00:08:13 return await fut
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 22, in run
Dec 28 00:08:13 result = self.fn(*self.args, **self.kwargs)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/asgiref/sync.py", line 486, in thread_handler
Dec 28 00:08:13 return func(*args, **kwargs)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Dec 28 00:08:13 return view_func(*args, **kwargs)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/django/views/generic/base.py", line 70, in view
Dec 28 00:08:13 return self.dispatch(request, *args, **kwargs)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
Dec 28 00:08:13 response = self.handle_exception(exc)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
Dec 28 00:08:13 self.raise_uncaught_exception(exc)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
Dec 28 00:08:13 raise exc
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
Dec 28 00:08:13 response = handler(request, *args, **kwargs)
Dec 28 00:08:13 File "/usr/lib/python3.10/contextlib.py", line 79, in inner
Dec 28 00:08:13 return func(*args, **kwds)
Dec 28 00:08:13 File "/app/code/backend/src/baserow/api/decorators.py", line 90, in func_wrapper
Dec 28 00:08:13 return func(*args, **kwargs)
Dec 28 00:08:13 File "/app/code/backend/src/baserow/api/user_files/views.py", line 61, in post
Dec 28 00:08:13 user_file = UserFileHandler().upload_user_file(request.user, file.name, file)
Dec 28 00:08:13 File "/app/code/backend/src/baserow/core/user_files/handler.py", line 257, in upload_user_file
Dec 28 00:08:13 storage.save(full_path, stream)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/django/core/files/storage.py", line 54, in save
Dec 28 00:08:13 name = self._save(name, content)
Dec 28 00:08:13 File "/app/code/backend/src/baserow/core/storage.py", line 8, in _save
Dec 28 00:08:13 return super()._save(name, content)
Dec 28 00:08:13 File "/app/code/env/lib/python3.10/site-packages/django/core/files/storage.py", line 279, in _save
Dec 28 00:08:13 fd = os.open(full_path, self.OS_OPEN_FLAGS, 0o666)
Dec 28 00:08:13 PermissionError: [Errno 13] Permission denied: '/app/data/media/user_files/sIjCc9JiqO2tnF3n7kcWdLaEJ3o1Geuv_a6e867bf79213cf4a792c688bfde862914c9faa44e1cb02e3c8c03ad89efc573.pdf'

[nebulon]

Was just looking into this and couldn't figure out where you were trying to upload files. I uploaded csv files for import and that worked fine at least, so I guess there is some other area on the app where one can upload other things?

[robi]

@nebulon the error points to the path below with a .pdf

[Errno 13] Permission denied: '/app/data/media/user_files/*.pdf

[midm]

@nebulon Yes there is a "file" column type on Baserow, to upload and store files. This is where I had this crash.

[robi]

@nebulon I can reproduce this on the demo instance.. when you create a new table, you get some default rows and columns, you then scroll right to add a new column, and choose file type, then try uploading any file. It will fail.

UPDATE:
The directories in /app/data/media are owned by root for some reason, chown -R to cloudron:cloudron does the trick.

Not sure why there is 62+MB of random images in user_files/, there's over a thousand files there!

Some images are back to root permissions after I did the chown to cloudron.

[nebulon]

Actually on a fresh install for me the uploads work just fine, also I can't quite see the permission issue. The start.sh has a chown -R cloudron:cloudron /app/data however as you indicated, something apparently changes ownership or creates files with root while the app is running.

Also all processes besides nginx are run as cloudron user https://git.cloudron.io/cloudron/baserow-app/-/blob/master/supervisor.conf

[robi]

@nebulon wow, not my experience at all with the demo instance.. it seemed like the baserow instance was doing things I didn't want it doing, like downloading 1000+ stock images (that's not what baserow is for!).

Did something nefarious sneak in there? You can inspect the app at https://brow.demo.cloudron.io/ (cloudron/cloudron)

It'd be nice to be able to see what all is running and changing permissions. Is there a way you can run execsnoop from startup?

Here's more info on execsnoop and how to log all processes being executed (it may not work with our new kernels though).

You can apparently follow a process using strace. If you know the PID of the process then you can do:

strace -o strace-<pid>.out -f -p <pid>

Notice the -f switch. It will help you to follow newly created processes that are descendants of the process whose PID was used in the command, above.

https://github.com/a2o/snoopy also seems to record process execution, aside from doing full audit process accounting (psacct).

[nebulon]

Hm really not sure how to reproduce this even in https://brow.demo.cloudron.io/database/13/table/70 (if that app instance still exists while others read this) I am able to upload files into the table just fine. PDFs as well as others, maybe something special about the file you try to upload?

[robi]

@nebulon that's because I fixed the permissions earlier, try a new instance and monitor the directory just as you chown

[girish]

I can confirm there is a whole bunch of files owned by root:

[girish]

I think the issue is that we do this sync_templates in the background:

echo "==> Syncing templates (in the background)"
/app/code/env/bin/python /app/code/backend/src/baserow/manage.py sync_templates &

And the above thing is also downloading a whole bunch of images from the web, not sure why.

[girish]

@robi said in baserow error on file column upload:

@nebulon wow, not my experience at all with the demo instance.. it seemed like the baserow instance was doing things I didn't want it doing, like downloading 1000+ stock images (that's not what baserow is for!).

Apparently, all those images are part of the "templates" - https://gitlab.com/bramw/baserow/-/tree/develop/backend/templates

[girish]

This is fixed in latest package.