GPX file upload error

[imc67]

When uploading a gpx file (import) you get a GUI error and in the logs it says:

Errno::EROFS (Read-only file system @ dir_s_mkdir - /app/code/storage/69):

[imc67]

I’ve found this info for the docker-compose, maybe it’s helps:

Volumes: Configure bind mounts for persistent data:

./public:/var/app/public
./watched:/var/app/tmp/imports/watched
./storage:/var/app/storage

Create the corresponding host folders: mkdir public watched storage. These should be empty before the first start.

https://deployn.de/en/blog/dawarich-install/

Or here:

https://github.com/Freika/dawarich/blob/master/docker/docker-compose.yml

[nebulon]

This should be fixed with latest package. I was able to upload and consume a .gpx file, however I couldn't figure out how to display the points in the map afterwards.

[imc67]

it works indeed. I found out that on the import page and imported file there are 2 tiny little icons, the left one shows the import on the map. However you can also go to the main page and select the date of the imported file.

I see you added a symlink to "storage", are the other two I mentioned not needed?

[imc67]

I have more than 10.000 gpx files to be imported and manually via the GUI there seems to be limits.

I found this, can you implement this "watched" folder also? https://dawarich.app/docs/tutorials/watcher

dawarich_app:
    image: freikin/dawarich:latest
    container_name: dawarich_app
    volumes:
      - public:/var/app/public
+     - watched:/var/app/tmp/imports/watched

  ...

  dawarich_sidekiq:
      image: freikin/dawarich:latest
      container_name: dawarich_sidekiq
      volumes:
        - public:/var/app/public
+       - watched:/var/app/tmp/imports/watched

[nebulon]

Did not even think of such a use-case, but I have never really worked with .gpx files as such. I will add the watcher directory also then

[nebulon]

So the app already has /app/code/tmp linked to /tmp/dawarich. So the imports/watched can already be used, though this is all in the /tmp mount. Not sure what upstream intends here, is one supposed to upload to a /tmp folder manually and how are things cleaned up there?

[imc67]

Mmm, when I go to Filemanager (of the app) I do see a folder “storage” but not “temp”, should I create it including “imports/watched”?

[imc67]

@nebulon said in GPX file upload error:

is one supposed to upload to a /tmp folder manually and how are things cleaned up there?

I’m not a programmer but my assumption is that the developer means by “temp” a folder that a user or the system can use for files that temporarily exist, either by ingress or whatever. I guess it can be mounted to /app/data/ so a user can get there?

Like in the docs, only the watched folder mapping to app/data?

volumes:
      - dawarich_public:/var/app/public
      - dawarich_watched:/var/app/tmp/imports/watched
      - dawarich_storage:/var/app/storage

[imc67]

I just noticed that “public” also needs to be symlinked, there are editable files but also the “export” folder needs to be writable. I guess as mentioned before all 3 needs to be symlinked to /app/data/?

[nebulon]

do you know what the public folder does?

[imc67]

@nebulon said in GPX file upload error:

do you know what the public folder does?

When I use Terminal of the app and go there I see some html files, images and export folder. I guess it’s all configurable to what you want to show in your browser.

[nebulon]

So we need to first understand what is supposed to be there, since content in /tmp will get regularily (though only after 10 days) purged from disk. Have to play more with the app to see what it expects

[nebulon]

So after reading through the code, it seems public/ is mounted as a volume probably to customize the page, however given that it also contains the code from the release, we should not make that writable, otherwise we have a hard time to update.

[nebulon]

The export also works fine for both .json and .gpx. Looks like the export/ folder is not actually used there.

I also found that the app stores the pid file in /tmp which will not go well when regularly purged, so I think we will move all that to /run instead.

[nebulon]

New package release links the watched folder from /app/data however I was not able to get the app actually consume anything from that folder, not even without the linking. So this may need more debugging in the app itself it seems.

[imc67]

@nebulon said in GPX file upload error:

I was not able to get the app actually consume anything from that folder, not even without the linking

I deleted the previous app completely and started fresh. The "consuming" from the folder is only once an hour if I remember correctly. Now testing ...

[imc67]

said in GPX file upload error:

@nebulon said in GPX file upload error:
Now testing ...

Test succeeded after 2 tries: in the watched folder you have to create a folder with the email address of the user, in that folder you can put ie. the gpx files. Exact on every whole hour they get fetched and processed. Now after 5 minutes they are still not removed, can’t find info on that.

[imc67]

update: the amount of gpx files I tried to load was either too much or simply didn't worked out. Together with ChatGPT I was able to consolidate (multiple-)day files to monthly files, kml files to gpx and when you max to 100.000 datapoint per gpx file (about 11MB) I was able to upload and process them via GUI.

The result is AMAZING, this Dawarich app is absolutely great for people, like me, who always has a tracker on his smartphone. It's bizarre to see all your travels, vacations, flights etc etc.

Next step: integrate Immich.

@nebulon thanks for your time and effort to publish this app, very happy with it!

The stats are also nice:

[luckow]

@imc67 What? Almost half the distance to the moon in 2023?