Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!

    Mastodon
    11
    34
    1860
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scooke
      scooke last edited by scooke

      This issue is only just now slowly popping up around the few forums I frequent. This post won't give all the juicy details, but suffice to say you need to be sure you have LOTS of room on your VPS if you are running Mastodon. My impression of Cloudron users is that we use high-RAM but not necessarily high-storage VPSes. My own Mastodon instance has been running for just over a year. I opened it up to federate with other instances bit by bit over time. At first 1GB of RAM and just a few 10Gs of space was needed. But as I connected with more and more federated instances, connected with relays... my RAM and space requirement shot up. Right now it is a lowly connected instance - I follow only 128 users, and have 26 followers, and have just 2 pinned #s, and it now requires no less than 5.5GB of RAM. I had to switch to a new Minio instance 3 days ago (I forgot to renew my precious KS-1 on which it lived) and in THREE days it has 20GB stored in the new Minio bucket.

      People, get ready.

      A life lived in fear is a life half-lived

      MikeHuntington doodlemania2 BrutalBirdie 3 Replies Last reply Reply Quote 4
      • MikeHuntington
        MikeHuntington @scooke last edited by

        @scooke Maybe I misread, but you say this is for a personal instance with just you as a user or do you have a small community using your instance? If so, how many users?

        scooke 1 Reply Last reply Reply Quote 0
        • doodlemania2
          doodlemania2 App Dev @scooke last edited by

          @scooke very accurate - even my single user instance consumes a lot of resources because I do lots of posting, and am federated with an open relay. Plan accordingly!

          1 Reply Last reply Reply Quote 0
          • scooke
            scooke @MikeHuntington last edited by

            @MikeHuntington Very similar to @doodlemania2 . It's just me and my instance STILL processes that much data. It's the federation-side which opens the doors to that much data. If I wasn't connected to relays, if I turned off federation, if I just posted for myself (but, that's missing the point of both Mastodon and federation/ActivityPub), then it would likely still consume less than a GB of RAM and have less than 20GB of data stored on it.

            A life lived in fear is a life half-lived

            1 Reply Last reply Reply Quote 0
            • BrutalBirdie
              BrutalBirdie Staff @scooke last edited by

              @scooke Thanks for the report!
              You think this would be doc worthy?

              Like my work? Consider donating a beer 🍻 Cheers!

              jdaviescoates 1 Reply Last reply Reply Quote 1
              • jdaviescoates
                jdaviescoates @BrutalBirdie last edited by

                @BrutalBirdie said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                You think this would be doc worthy?

                Can't hurt to mention it somewhere in the docs (and maybe even during the install process) that once you (or other users) start following a few people, and with federation on (it is by default) Mastodon will quickly start taking up a A LOT of space.

                I've not yet noticed the RAM side of things myself, but that's possibly just because as soon as I noticed the disk space had run out and I unfollowed everyone as soon as I got the server running again to buy myself a little time to migrate over to a server with more storage.

                I use Cloudron with Gandi & Hetzner

                MikeHuntington 1 Reply Last reply Reply Quote 0
                • MikeHuntington
                  MikeHuntington @jdaviescoates last edited by

                  There's also an option in Mastodon to purge remote media (pulled in with federation on). Not sure if that config option is in Cloudron, but that would help with space.

                  nebulon 1 Reply Last reply Reply Quote 1
                  • nebulon
                    nebulon Staff @MikeHuntington last edited by

                    @MikeHuntington currently the package runs that purging as a cronjob: https://git.cloudron.io/cloudron/mastodon-app/-/blob/master/cleanup.sh

                    Is that the trigger you had in mind?

                    MikeHuntington 1 Reply Last reply Reply Quote 2
                    • MikeHuntington
                      MikeHuntington @nebulon last edited by

                      @nebulon Yup that's it! Good to see it was already a scheduled task.

                      By the way, today my database filled up and I was no longer able to access my Mastodon app. (using Digital Ocean) I increased my droplet size, but that didn't seem to help as the database is in a docker image(?). Seems there wasn't a way to resize that image to use more space.

                      So my app is stuck is a state of database failing to connect because space is maxed out.

                      nebulon 1 Reply Last reply Reply Quote 0
                      • nebulon
                        nebulon Staff @MikeHuntington last edited by

                        @MikeHuntington databases are provided as services to the apps on Cloudron. They run in their own container. Did the database run out of disk space (by default those are stored in /home/yellowtent/...) or did it run out of memory? For the latter you can adjust the memory limit in the services view.

                        MikeHuntington 1 Reply Last reply Reply Quote 0
                        • MikeHuntington
                          MikeHuntington @nebulon last edited by

                          @nebulon It ran out of diskspace after running mastodon for about 24 hours (with federation relays added)

                          nebulon 1 Reply Last reply Reply Quote 0
                          • nebulon
                            nebulon Staff @MikeHuntington last edited by

                            @MikeHuntington in such cases go through the checklist at https://docs.cloudron.io/troubleshooting/#recovery-after-disk-full

                            MikeHuntington 1 Reply Last reply Reply Quote 1
                            • MikeHuntington
                              MikeHuntington @nebulon last edited by

                              @nebulon Thank you for that link! Is if possible to have my database run on a different server than the one cloudron is running on? Also is AWS S3 an option?

                              I know I can edit Mastodon's .env file directly but will Cloudron support that?

                              I think that'll help with diskspace a ton as well.

                              scooke 1 Reply Last reply Reply Quote 0
                              • scooke
                                scooke @MikeHuntington last edited by scooke

                                @MikeHuntington If you check out Cloudron's Mastodon forum there are a few discussions about how to use software like S3, Minio, Object Storage. It will def work.
                                EDIT: I just restarted my Owncast instance and set it up to use my Minio instance in just a few minutes. Once you get your head around how it all works, it's super smooth and do-able.

                                A life lived in fear is a life half-lived

                                1 Reply Last reply Reply Quote 2
                                • robi
                                  robi last edited by

                                  rclone is a swiss army tool for connecting/mounting to remote storage and more.

                                  So if you have a deep store in GDrive, B2, Scaleway, Dropbox, etc you can connect them to your apps data/cache dirs.

                                  Life of Advanced Technology

                                  L 1 Reply Last reply Reply Quote 2
                                  • scooke
                                    scooke last edited by

                                    @MikeHuntington True, and eventually I'll implement those commands to run automatically, but I imagine that someone who is surprised and frustrated by a rapidly-filling-up server won't have the tech skills, inclination or time to run those regularly anyway.

                                    A win for Cloudron would be to offer a combined federation app - Mastodon + Scaleway, where the Mastodon instance also generates a user, Access Key ID and Secret Access Key which the user could then plug into a Scaleway Object Storage bucket (or there could be a drop-own menu offering S3, Scaleway or Minio), and also set the .env.production settings correctly. I realize doing that is relatively simple if you've done it before, but otherwise it could prove to be a barrier.

                                    A life lived in fear is a life half-lived

                                    doodlemania2 L jdaviescoates 3 Replies Last reply Reply Quote 1
                                    • doodlemania2
                                      doodlemania2 App Dev @scooke last edited by

                                      @scooke That's a fun idea!

                                      1 Reply Last reply Reply Quote 1
                                      • L
                                        LoudLemur @robi last edited by

                                        @robi said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                        rclone is a swiss army tool for connecting/mounting to remote storage and more.

                                        So if you have a deep store in GDrive, B2, Scaleway, Dropbox, etc you can connect them to your apps data/cache dirs.

                                        Thanks for this very sweet knowledge! rclone has a (currently experimental) GUI (Graphical User Interface) which appears in the web browser. If Cloudron try supporting this, I hope they make use of it.

                                        https://rclone.org/gui/

                                        There are some lovely things rclone can interact with, apparently. I notice that rclone can help with internet archive:

                                        #https://rclone.org/internetarchive/

                                        1 Reply Last reply Reply Quote 0
                                        • L
                                          LoudLemur @scooke last edited by

                                          @scooke said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                          @MikeHuntington True, and eventually I'll implement those commands to run automatically, but I imagine that someone who is surprised and frustrated by a rapidly-filling-up server won't have the tech skills, inclination or time to run those regularly anyway.

                                          A win for Cloudron would be to offer a combined federation app - Mastodon + Scaleway, where the Mastodon instance also generates a user, Access Key ID and Secret Access Key which the user could then plug into a Scaleway Object Storage bucket (or there could be a drop-own menu offering S3, Scaleway or Minio), and also set the .env.production settings correctly. I realize doing that is relatively simple if you've done it before, but otherwise it could prove to be a barrier.

                                          This is a great idea and could remove a lot of friction for people wanting to self-host in the Fediverse.

                                          Sam_uk 1 Reply Last reply Reply Quote 0
                                          • Sam_uk
                                            Sam_uk @LoudLemur last edited by

                                            @LoudLemur said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                            A win for Cloudron would be to offer a combined federation app - Mastodon + Scaleway, where the Mastodon instance also generates a user, Access Key ID and Secret Access Key which the user could then plug into a Scaleway Object Storage bucket

                                            +1 I'm too scared to run Mastodon on my server due to it eating all the disk. This would help.

                                            jdaviescoates 1 Reply Last reply Reply Quote 0
                                            • jdaviescoates
                                              jdaviescoates @Sam_uk last edited by

                                              @Sam_uk said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                              I'm too scared to run Mastodon on my server due to it eating all the disk. This would help.

                                              Could just spin up a new VPS for it (that's what I did, and switfly ran out of space! But then increased the size...)

                                              And/ or just immediately go an set-up some storage elsewhere as a Volume and set-up the app to use that volume as the data storage directory as per https://docs.cloudron.io/apps/#data-directory

                                              I use Cloudron with Gandi & Hetzner

                                              L 1 Reply Last reply Reply Quote 1
                                              • L
                                                LoudLemur @jdaviescoates last edited by

                                                @jdaviescoates said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                                @Sam_uk said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                                I'm too scared to run Mastodon on my server due to it eating all the disk. This would help.

                                                Could just spin up a new VPS for it (that's what I did, and switfly ran out of space! But then increased the size...)

                                                And/ or just immediately go an set-up some storage elsewhere as a Volume and set-up the app to use that volume as the data storage directory as per https://docs.cloudron.io/apps/#data-directory

                                                People like you make this place the brilliant community it is! Thanks.

                                                1 Reply Last reply Reply Quote 2
                                                • jdaviescoates
                                                  jdaviescoates @scooke last edited by

                                                  @scooke said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                                  A win for Cloudron would be to offer a combined federation app - Mastodon + Scaleway, where the Mastodon instance also generates a user, Access Key ID and Secret Access Key which the user could then plug into a Scaleway Object Storage bucket (or there could be a drop-own menu offering S3, Scaleway or Minio), and also set the .env.production settings correctly. I realize doing that is relatively simple if you've done it before, but otherwise it could prove to be a barrier.

                                                  +1 something like that would be great because I've just gone to create a bucket on Scaleway (easy enough), but now I'm at a loss as to how I actually connect that to a volume. Which type of mount point should I use for the Volume?

                                                  I use Cloudron with Gandi & Hetzner

                                                  robi 1 Reply Last reply Reply Quote 0
                                                  • robi
                                                    robi @jdaviescoates last edited by

                                                    @jdaviescoates you'd have to do that through rclone manually. Cloudron only supports S3 as a backup target, not App volumes.

                                                    Life of Advanced Technology

                                                    jdaviescoates 1 Reply Last reply Reply Quote 1
                                                    • jdaviescoates
                                                      jdaviescoates @robi last edited by

                                                      @robi sounds like maybe @scooke has managed it previously?

                                                      I use Cloudron with Gandi & Hetzner

                                                      1 Reply Last reply Reply Quote 0
                                                      • Referenced by  girish girish 
                                                      • Referenced by  chetbaker chetbaker 
                                                      • dxciBel
                                                        dxciBel last edited by

                                                        So I've been running into this exact issue really soon after hosting an instance with ~30 users for a few weeks. I'm now trying to migrate media files to an S3 storage, in my case it's a Linode Object Storage.

                                                        I've found this awesome guide which makes things pretty clear, but there's a question that came up when I read through it. In the section about configuring the bucket it says:

                                                        Also, these instructions are specific to manual deployments, you may need to modify paths slightly for docker or other automatic deployments.

                                                        Just to make sure, and please excuse if this question seems kinda inane, but: How exactly do I migrate my media files via Cloudron? Do I do it via the server console (Linode) or the Cloudron terminal? Is there anything extra I need to be aware of, any instructions that differ from this GitHub page?

                                                        Just trying to make sure I don't break everything while trying to migrate to external media storage. Please be patient. 🙏

                                                        scooke 1 Reply Last reply Reply Quote 2
                                                        • scooke
                                                          scooke @dxciBel last edited by scooke

                                                          @dxciBel I might be wrong, and I hope I am for your sake, but when you change storage systems I think you will basically be starting over. There is a way to move buckets if origin and destination are both already S3. For me, even though I was one user on my own instance, at the time the amount of data I had stored was huge, so I just called it a loss, changed to Minio, and let the instance slowly repopulate all the data it needed. The one thing I did do was SAVE all my own images, headers, icons, etc., for my instance.

                                                          In the process of writing this I did find these:

                                                          https://stanislas.blog/2018/05/moving-mastodon-media-files-to-wasabi-object-storage/
                                                          - the author recommends AGAINST Wasabi, but you want to read about the tools to move data from a regular storage to S3 using aws-cli. Almost halfway down there is this command which is what you want, and which you'll (obviously) have to adjust for your use-case: aws s3 sync public/system/ s3://my-bucket/ --endpoint-url=https://s3.wasabisys.com. "aws" is the command, "s3" is an option, "public/system" refers to where the data currently is, in your cloudron image, and "s3://my-bucket" is your Minio instance. I guess the last bit you need to include; if it doesn't work initially tweak the endpoint a few times, like remove the https...not sure how that it will go, but, it looks straightforward.

                                                          Here are a few more. I suggest reading all of them before embarking on your journey:
                                                          https://github.com/cybrespace/cybrespace-meta/blob/master/s3.md
                                                          https://chrishubbs.com/2022/11/19/hosting-a-mastodon-instance-moving-asset-storage-to-s3/

                                                          A life lived in fear is a life half-lived

                                                          jdaviescoates dxciBel 2 Replies Last reply Reply Quote 4
                                                          • jdaviescoates
                                                            jdaviescoates @scooke last edited by jdaviescoates

                                                            @scooke said in Running a federated Mastodon instance will take up ALOT of space and RAM - be prepared!:

                                                            Almost halfway down there is this command which is what you want, and which you'll (obviously) have to adjust for your use-case: aws s3 sync public/system/ s3://my-bucket/ --endpoint-url=https://s3.wasabisys.com. "aws" is the command, "s3" is an option, "public/system" refers to where the data currently is, in your cloudron image, and "s3://my-bucket" is your Minio instance. I guess the last bit you need to include; if it doesn't work initially tweak the endpoint a few times, like remove the https...not sure how that it will go, but, it looks straightforward.

                                                            Yeah, I've not attempted moving files but it's obviously possible.

                                                            She also this guide that has similar command to above.

                                                            https://thomas-leister.de/en/mastodon-s3-media-storage/

                                                            Indeed in their set-up they not only move stuff over but also do some clever nginx stuff to serve files locally if they exist and if not to fetch from S3 and cache locally.

                                                            I use Cloudron with Gandi & Hetzner

                                                            doodlemania2 shanelord01 2 Replies Last reply Reply Quote 0
                                                            • doodlemania2
                                                              doodlemania2 App Dev @jdaviescoates last edited by

                                                              @jdaviescoates I can +1 the need for a cache - moving to S3 works great but it does slow things way down. I am trying to find a way to do a local nginx cache with CR but coming up short so far.

                                                              1 Reply Last reply Reply Quote 1
                                                              • shanelord01
                                                                shanelord01 @jdaviescoates last edited by

                                                                @jdaviescoates It doesn't seem possible to change the NGINX settings for this to work in Cloudron, unless I'm missing something?

                                                                jdaviescoates 1 Reply Last reply Reply Quote 1
                                                                • jdaviescoates
                                                                  jdaviescoates @shanelord01 last edited by

                                                                  @shanelord01 I've no idea to be honest, but it would be nice if it was possible.

                                                                  @doodlemania2 may work it out, or perhaps @staff can assist

                                                                  I use Cloudron with Gandi & Hetzner

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • dxciBel
                                                                    dxciBel @scooke last edited by dxciBel

                                                                    @scooke As it turns out, with the help of the blog post you found, it was possible. Moving storage to S3 was rather easy, you just have to add the information to env.production. I moved the files to the bucket with the aws tool mentioned in the linked post, but s3cmd would most likely work as well. Last hurdle was making the bucket publically accessible since all the copied files are private by default. Made a policy.json file using this support doc from Linode as an example and voila, everything works again.

                                                                    doodlemania2 1 Reply Last reply Reply Quote 1
                                                                    • doodlemania2
                                                                      doodlemania2 App Dev @dxciBel last edited by

                                                                      Yeah, the same doc shows how to set up an Ngnix cache - I'm gonna try to hack something together to see if I can front that in CR somehow. Could serve some other generic purposes too for other systems that could benefit from an S3 cache.

                                                                      robi 1 Reply Last reply Reply Quote 1
                                                                      • robi
                                                                        robi @doodlemania2 last edited by

                                                                        @doodlemania2 Seaweed FS may be what you're looking for to cache/gateway object storage.

                                                                        https://forum.cloudron.io/post/56024

                                                                        Life of Advanced Technology

                                                                        1 Reply Last reply Reply Quote 0
                                                                        • First post
                                                                          Last post
                                                                        Powered by NodeBB