Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Unsolved Profile images doesnt load on S3

    Pixelfed
    6
    22
    254
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ehsan0921
      ehsan0921 last edited by girish

      I raise this issue in GitHub:

      Hi guys

      I linked S3 as below.

      PF_ENABLE_CLOUD=true
      FILESYSTEM_DRIVER=s3
      FILESYSTEM_CLOUD=s3
      AWS_ACCESS_KEY_ID=******
      AWS_SECRET_ACCESS_KEY=*******
      AWS_DEFAULT_REGION=ap-southeast-2
      AWS_BUCKET=********

      everything is fine with post. all uploading on S3 with no issue.
      but when it comes to profile photo change. it doesn't work.
      seems the address have some issue.

      the address is like below:
      https://aust.socialhttps//austsocial.s3.ap-southeast-2.amazonaws.com/public/*************

      but it should be line
      https//austsocial.s3.ap-southeast-2.amazonaws.com/public/*************

      can anyone help me with that?


      The answer I think is here.
      https://github.com/pixelfed/pixelfed/commit/672f7c8c

      but
      how to access the profile.php file via cloudron or FileZilla?
      where are the files??

      jdaviescoates 2 Replies Last reply Reply Quote 1
      • nebulon
        nebulon Staff last edited by

        On Cloudron you can only change those code files temporarily when recovery mode is enabled for the app. This also means you have to run the app manually through a terminal into the app container. https://docs.cloudron.io/troubleshooting/#unresponsive-app

        Once upstream makes a new release with the fix, we will update the package of course.

        jdaviescoates 1 Reply Last reply Reply Quote 1
        • jdaviescoates
          jdaviescoates @ehsan0921 last edited by jdaviescoates

          @ehsan0921 said in Profile images dosnt load on S3:

          I raise this issue in GitHub:

          Hi guys

          I linked S3 as below.

          PF_ENABLE_CLOUD=true
          FILESYSTEM_DRIVER=s3
          FILESYSTEM_CLOUD=s3
          AWS_ACCESS_KEY_ID=******
          AWS_SECRET_ACCESS_KEY=*******
          AWS_DEFAULT_REGION=ap-southeast-2
          AWS_BUCKET=********

          everything is fine with post. all uploading on S3 with no issue.
          but when it comes to profile photo change. it doesn't work.
          seems the address have some issue.

          Hi @ehsan0921 (or anyone else who has this working) do you think you could share more about your Pixelfed S3 set-up?

          I'm trying to get it set-up with my Pixelfed install too but struggling to get it to work.

          You don't appear to have specified any AWS_URL nor AWS_ENDPOINT but I'm presuming you must have at least one of those in your config too, right?

          I've currently got this in my env.production

          # Trying to store media on Scaleway S3 object storage
          PF_ENABLE_CLOUD=true
          FILESYSTEM_DRIVER=s3 
          FILESYSTEM_CLOUD=s3
          PF_LOCAL_AVATAR_TO_CLOUD=true
          MEDIA_DELETE_LOCAL_AFTER_CLOUD=true
          AWS_ACCESS_KEY_ID=xxx
          AWS_SECRET_ACCESS_KEY=xxx
          AWS_DEFAULT_REGION=fr-par
          AWS_BUCKET=bucket-name
          AWS_URL=https://bucket-name.s3.fr-par.scw.cloud
          AWS_ENDPOINT=s3.fr-par.scw.cloud
          AWS_USE_PATH_STYLE_ENDPOINT=false
          

          But it's not working.

          I see in the logs:

          Jan 22 13:18:46 [previous exception] [object] (GuzzleHttp\\Exception\\ConnectException(code: 0): cURL error 6: Could not resolve host: bridport-pics. (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) at /app/code/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:200)
          

          @scooke have you done this with Pixelfed too?

          jdaviescoates 1 Reply Last reply Reply Quote 0
          • jdaviescoates
            jdaviescoates @jdaviescoates last edited by jdaviescoates

            This post is deleted!
            1 Reply Last reply Reply Quote 0
            • jdaviescoates
              jdaviescoates last edited by

              @jdaviescoates said in Profile images dosnt load on S3:

              But it's not working.

              It is now! 😄

              I looked my settings for Mastodon for some clues and noticed that in there the endpoint included https://

              All I had to do was to add that and then it started working! I wish there were most established standards around this. Some apps seem to want/ need the https:// in the endpoint and others don't so it become a matter of trail and error.

              This is my working config in env.production:

              # Store media on Scaleway S3 object storage
              PF_ENABLE_CLOUD=true
              FILESYSTEM_DRIVER=s3 
              FILESYSTEM_CLOUD=s3
              PF_LOCAL_AVATAR_TO_CLOUD=true
              MEDIA_DELETE_LOCAL_AFTER_CLOUD=true
              AWS_ACCESS_KEY_ID=xxx
              AWS_SECRET_ACCESS_KEY=xxx
              AWS_DEFAULT_REGION=fr-par
              AWS_BUCKET=bucket-name
              AWS_URL=https://bucket-name.s3.fr-par.scw.cloud
              AWS_ENDPOINT=https://s3.fr-par.scw.cloud
              AWS_USE_PATH_STYLE_ENDPOINT=false
              
              1 Reply Last reply Reply Quote 0
              • jdaviescoates
                jdaviescoates @ehsan0921 last edited by

                @ehsan0921 said in Profile images dosnt load on S3:

                but when it comes to profile photo change. it doesn't work.

                Ah, but I'm also hitting this issue.

                Did you manage to get it solved @ehsan0921 ?

                1 Reply Last reply Reply Quote 0
                • jdaviescoates
                  jdaviescoates @nebulon last edited by

                  @nebulon said in Profile images dosnt load on S3:

                  Once upstream makes a new release with the fix, we will update the package of course.

                  I'm confused though, because that fix was committed back in July 2022 and is tagged Pixelfed 0.11.4 which is also what we're running on Cloudron, so I'm unclear why Profile pics with S3 enabled aren't working?

                  jdaviescoates 1 Reply Last reply Reply Quote 0
                  • jdaviescoates
                    jdaviescoates @jdaviescoates last edited by jdaviescoates

                    Aha looking at the link for the avatar image, I see it's like this:

                    https://bridport.picshttps//bridport-pics.s3.fr-par.scw.cloud/public/avatars/052/274/889/869/491/404/9/3KE8LIMTBRbT8dlgbysN_avatar.png?v=3

                    If I take of off the superflous https://bridport.pics at the beginning it works fine:

                    https://bridport-pics.s3.fr-par.scw.cloud/public/avatars/052/274/889/869/491/404/9/3KE8LIMTBRbT8dlgbysN_avatar.png?v=3

                    So I wonder how/ why that extra bit is coming from...

                    I can see this in the logs:

                    Jan 22 13:59:19 [2023-01-22 13:59:18] production.ERROR: Image source not readable {"exception":"[object] (Intervention\\Image\\Exception\\NotReadableException(code: 0): Image source not readable at /app/code/vendor/intervention/image/src/Intervention/Image/AbstractDecoder.php:351)
                    

                    So I don't know if this is a bug or if I've got something wrong in my config?

                    jdaviescoates 1 Reply Last reply Reply Quote 0
                    • jdaviescoates
                      jdaviescoates @jdaviescoates last edited by

                      @jdaviescoates said in Profile images dosnt load on S3:

                      So I don't know if this is a bug or if I've got something wrong in my config?

                      Reading this https://github.com/pixelfed/pixelfed/issues/4041

                      It sounds like you have to have FILESYSTEM_DRIVE=local for the Avatars to work. So I tried that.

                      But now it seems all files are getting stored twice, once locally and then again in the Object Storage.

                      See e.g. https://bridport.pics/i/web/post/522792891553329155

                      Really I only want to store thing in the object storage, not locally at all. But I've got MEDIA_DELETE_LOCAL_AFTER_CLOUD=true so maybe eventually it will delete the local versions? 🤷

                      Yep, it seems that perhaps files are just being cached locally, because if I open the Web Terminal and run php artisan cache:clear then right-click Open image in new tab it opens the S3 version (although as you can see in the post linked to above, for now the local version still seems to exist too... presumably eventually that'll get deleted, ah but then again perhaps not as per https://github.com/pixelfed/pixelfed/issues/3845#issuecomment-1331691661 ).

                      jdaviescoates 1 Reply Last reply Reply Quote 0
                      • jdaviescoates
                        jdaviescoates @jdaviescoates last edited by

                        @jdaviescoates said in Profile images dosnt load on S3:

                        presumably eventually that'll get deleted, ah but then again perhaps not as per https://github.com/pixelfed/pixelfed/issues/3845#issuecomment-1331691661 ).

                        I've filed an issue about this:

                        https://github.com/pixelfed/pixelfed/issues/4114

                        I've also confirmed that manually deleting images from the local storage via the File Manager does not break posts.

                        doodlemania2 1 Reply Last reply Reply Quote 1
                        • doodlemania2
                          doodlemania2 App Dev @jdaviescoates last edited by

                          @jdaviescoates similar impact on the latest build since we switched to dev branch! cc: @girish FYI only. Hope they get that bug fixed 😉

                          jdaviescoates 2 Replies Last reply Reply Quote 1
                          • jdaviescoates
                            jdaviescoates @doodlemania2 last edited by jdaviescoates

                            @doodlemania2 yeah, it seems at present if you want profile images/ avatars to work you HAVE to have FILESYSTEM_DRIVER=local but then all your media is store BOTH locally AND on S3. Odd.

                            The obvious solution would seem to me to be to simply make avatars/ profile images work when FILESYSTEM_DRIVER=s3 too, then all would be fine and dandy imho.

                            doodlemania2 1 Reply Last reply Reply Quote 0
                            • doodlemania2
                              doodlemania2 App Dev @jdaviescoates last edited by

                              @jdaviescoates yep - it's a bit of an odd situation. and blocking me from going live 😞

                              jdaviescoates 1 Reply Last reply Reply Quote 0
                              • jdaviescoates
                                jdaviescoates @doodlemania2 last edited by

                                @doodlemania2 said in Profile images dosnt load on S3:

                                Hope they get that bug fixed

                                Like it to help them pay attention to is? 😉

                                1 Reply Last reply Reply Quote 0
                                • jdaviescoates
                                  jdaviescoates @doodlemania2 last edited by

                                  @doodlemania2 said in Profile images dosnt load on S3:

                                  @jdaviescoates yep - it's a bit of an odd situation. and blocking me from going live 😞

                                  I'm not expecting to be swamped with users and so I've decided to go live with having everything store twice for now because I've a fair bit of unused disk space right now. If I end up getting a load of active users then I may need to manually start deleting stuff from local storage until this is fixed, but hopefully it'll be fixed before then!

                                  doodlemania2 1 Reply Last reply Reply Quote 0
                                  • doodlemania2
                                    doodlemania2 App Dev @jdaviescoates last edited by

                                    @jdaviescoates I'm stuck on the URL still being munged with half local and half s3 with the profile pic...

                                    jdaviescoates 1 Reply Last reply Reply Quote 0
                                    • jdaviescoates
                                      jdaviescoates @doodlemania2 last edited by

                                      @doodlemania2 said in Profile images doesnt load on S3:

                                      @jdaviescoates I'm stuck on the URL still being munged with half local and half s3 with the profile pic...

                                      Ah, I think removing/ commenting out AWS_URL might fix that.

                                      Can't remember exactly but I had it and then later after doing something, I didn't.

                                      I think perhaps it was that, or it that plus changing FILESYSTEM_DRIVER from s3 to local.

                                      doodlemania2 1 Reply Last reply Reply Quote 1
                                      • doodlemania2
                                        doodlemania2 App Dev @jdaviescoates last edited by

                                        @jdaviescoates filesystem to local did it!

                                        1 Reply Last reply Reply Quote 2
                                        • A
                                          alwynispat last edited by

                                          Another weird thing about the app is that if you upload from web, it's serving the image from the Server even though both locations has it.
                                          if you upload from the app, it'll serve from s3.

                                          then when you run php artisan cache:clear it'll get fixed.

                                          Wonder if it'll auto switch to s3 without touching.

                                          Since it's storing images on both locations, will it like auto delete from server?

                                          Find me at Mastodon

                                          jdaviescoates 1 Reply Last reply Reply Quote 0
                                          • jdaviescoates
                                            jdaviescoates @alwynispat last edited by

                                            @alwynispat said in Profile images doesnt load on S3:

                                            Wonder if it'll auto switch to s3 without touching.
                                            Since it's storing images on both locations, will it like auto delete from server?

                                            I think it's supposed to do that after 12 hours or something , but I've not checked properly to see if that is indeed the case .

                                            1 Reply Last reply Reply Quote 1
                                            • G
                                              gh0stface last edited by

                                              Hello, is there any update/guidance from the Cloudron team on this? As in the original topic posted- profile images not properly working if using S3 as storage?

                                              Anyone else able to figure anything else out with the profile avatar and using S3 compatible storage?

                                              If I'm understanding this correctly (and I could be wrong), this issue was addressed by this commit back in July 2022. And was bundled into the v0.11.4 release, which was made available in Oct 2022.

                                              A 1 Reply Last reply Reply Quote 0
                                              • A
                                                alwynispat @gh0stface last edited by

                                                @gh0stface so this is what I've setup;

                                                ## S3 Storage
                                                PF_ENABLE_CLOUD=true
                                                FILESYSTEM_DRIVER=local
                                                FILESYSTEM_CLOUD=s3
                                                PF_LOCAL_AVATAR_TO_CLOUD=true
                                                MEDIA_DELETE_LOCAL_AFTER_CLOUD=true
                                                AWS_ACCESS_KEY_ID=xxx
                                                AWS_SECRET_ACCESS_KEY=xxx
                                                AWS_DEFAULT_REGION=xxx
                                                AWS_BUCKET=mediapixelfedsg
                                                AWS_URL=https://media.pixelfed.sg
                                                AWS_ENDPOINT=https://xxx.backblazeb2.com
                                                #AWS_USE_PATH_STYLE_ENDPOINT=false
                                                

                                                and it works.

                                                Find me at Mastodon

                                                1 Reply Last reply Reply Quote 2
                                                • Topic has been marked as a question  nebulon nebulon 
                                                • First post
                                                  Last post
                                                Powered by NodeBB