Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved How to rsync files into surfer

    Surfer
    4
    9
    394
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jdaviescoates
      jdaviescoates last edited by

      My friends who've been hosting library.uniteddiversity.coop for years want to downscale their server and so I need to find it a new home... My plan is to just use Surfer, but I'm struggling to work out how to rsync my files?

      I've been trying locally mounting Surfer using Davfs to /home/josef/Library2/ and then doing this:

      rsync -avrh josef@library.uniteddiversity.coop:/data/josef/library.uniteddiversity.coop/ /home/josef/Library2/
      

      But it doesn't have the permissions to mkdir.

      I can SSH into library.uniteddiversity.coop

      I'm guess there must be a simple way to do this but I'm not sure what it is?!? Help!

      Thanks,

      Josef.

      I use Cloudron with Gandi & Hetzner

      1 Reply Last reply Reply Quote 0
      • nebulon
        nebulon Staff last edited by

        I think there are two things here. Rsync as such would only work via the SSH connection into the host system. For that you need to find the surfer instance data folder. Unless changed, this is at /home/yellowtent/appsdata/<appid>/data/public

        For the webdav mount, creating directories should work fine, I use a webdav mount (although via gnome filemanager) all the time. Do you have any logs or how did you setup the mountpoint?

        jdaviescoates 1 Reply Last reply Reply Quote 0
        • jdaviescoates
          jdaviescoates @nebulon last edited by

          @nebulon said in How to rsync files into surfer:

          I think there are two things here. Rsync as such would only work via the SSH connection into the host system. For that you need to find the surfer instance data folder. Unless changed, this is at /home/yellowtent/appsdata/<appid>/data/public

          Hmz, OK, so I've found the appid

          But now I'm unsure of what command I should try.

          I just tried:

          rsync -avrh josef@library.uniteddiversity.coop:/data/josef/library.uniteddiversity.coop/ jdaviescoates@library2.uniteddiversity.com:/home/yellowtent/appsdata/e815de83-e018-4507-be2d-e3d77d818aae/data/public
          

          But got:

          The source and destination cannot both be remote.
          

          So I guess I need to SSH into library.uniteddiversity.coop and then run the command there...

          So I tried:

          josef@library:~$ rsync -avrhP /data/josef/library.uniteddiversity.coop/ jdaviescoates@library2.uniteddiversity.coop:/home/yellowtent/appsdata/e815de83-e018-4507-be2d-e3d77d818aae/data/public
          The authenticity of host 'library2.uniteddiversity.coop (116.203.252.254)' can't be established.
          ECDSA key fingerprint is f4:12:64:cd:e0:bd:2e:e3:39:24:ad:17:98:13:c2:a2.
          Are you sure you want to continue connecting (yes/no)? yes 
          Warning: Permanently added 'library2.uniteddiversity.coop,116.203.252.254' (ECDSA) to the list of known hosts.
          jdaviescoates@library2.uniteddiversity.coop's password: 
          Permission denied, please try again.
          

          Seems like I'm getting there, but for some reason it's not accepting my pw (I'm copy/ pasting of bitwarden, but maybe the pasting isn't working or something?)

          @nebulon said in How to rsync files into surfer:

          For the webdav mount, creating directories should work fine, I use a webdav mount (although via gnome filemanager) all the time. Do you have any logs or how did you setup the mountpoint?

          At first I just went to + Other Location and used the Connect to Server field and the bottom of the screen.

          Then I just dragged of the files from one gnome filemanager tab to the other. This seemed to mostly work fine aside from the fact I kept getting those symlink errors, and then some other errors too, like e.g.

          Screenshot from 2020-12-02 23-14-39.png

          It was because I kept getting various errors (which stopped the copying process until I chose Skip) that I decided to try the rsync route.

          To try that I decided to try mounting the Surfer app using:

          sudo mount -t davfs https://library2.uniteddiversity.coop/_webdav/ /home/josef/Library2/
          

          I think I tried doing it without sudo first but it wouldn't let me.

          But I've realised that is why I can't mkdir, because /home/josef/Library2/ is owned by root.

          So I just ran:

          sudo chown -R josef /home/josef/Library2/
          

          Which seemed to fix that issue, although I did also get this:

          chown: changing ownership of '/home/josef/Library2/lost+found': Invalid argument
          

          Anyways, I ignored that for now and tried my previous rsync command:

          rsync -avrh josef@library.uniteddiversity.coop:/data/josef/library.uniteddiversity.coop/ /home/josef/Library2/
          

          And this now seems to be working! 😄

          Although it is still having issues with the symlinks:

          rsync: symlink "/home/josef/Library2/Community_Currencies" -> "/data/josef/library.uniteddiversity.coop/Money_and_Economics/Community_Currencies" failed: Function not implemented (38)
          rsync: symlink "/home/josef/Library2/Cooperatives" -> "/data/josef/library.uniteddiversity.coop/Money_and_Economics/Cooperatives" failed: Function not implemented (38)
          

          (and I forgot to add P to the -avrh to show me the progress and help restart partial it something happens mid-way)...

          So I seem to managed to do this now - yay!

          But there is still the symlink issue.

          I use Cloudron with Gandi & Hetzner

          1 Reply Last reply Reply Quote 0
          • jeau
            jeau App Dev last edited by

            On the other hand, is ther a way to run a cron job inside a Surfer app?

            nebulon 1 Reply Last reply Reply Quote 0
            • nebulon
              nebulon Staff @jeau last edited by

              @jeau there is no way to do so currently. What would be the use-case for that?

              jeau 1 Reply Last reply Reply Quote 1
              • jeau
                jeau App Dev @nebulon last edited by

                @nebulon I have an ongoing project with two requests from the team:

                • automatically generate a catalog from remote files,
                • get an updated static site on github

                I can easily make a script to do this inside the Surfer app if this script can be launched periodically

                T 1 Reply Last reply Reply Quote 1
                • T
                  thetomester13 App Dev @jeau last edited by

                  @jeau I think a way to do this would be to have a cron running on a different system (even on a LAMP app maybe) and use the surfer CLI to push files.

                  nebulon 1 Reply Last reply Reply Quote 1
                  • nebulon
                    nebulon Staff @thetomester13 last edited by

                    @thetomester13 right I also think this does not directly belong to the surfer app. It could maybe also more easily achieved through some git hooks or ci/cd pipelines.

                    jeau 1 Reply Last reply Reply Quote 2
                    • jeau
                      jeau App Dev @nebulon last edited by

                      @nebulon @thetomester13 I think you're right, there are several ways to do it

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post
                      Powered by NodeBB