Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Run Additional Sidekiqs

    Mastodon
    3
    6
    126
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • doodlemania2
      doodlemania2 App Dev last edited by girish

      I think I may have gotten to a threshold of sorts. The sidekiq running in my Mastodon instance isn't able to keep up. Top shows it running in the container no problem, but apparently I need more than one instance to get some parallelism rocking. Top also showed the container wasn't pegged at 100% CPU, so it had room to grow. I also bumped up the Resources from default 50% to 75%.

      So, I ended up with about a 6 hr backlog (around 40,000 messages) in my queues. In order to get them to clear, I opened up two terminal instances and manually ran bundle exec sidekiq in each and that chewed through them pretty quickly. Once the queues were drained, I closed them out but it started building back up again - the default sidekiq is definitely running, just a lot slower than needed.

      So, wondering if we could have a config option to ramp additional sidekiqs up?

      jdaviescoates 1 Reply Last reply Reply Quote 2
      • jdaviescoates
        jdaviescoates @doodlemania2 last edited by

        @doodlemania2 have you seen:

        https://docs.cloudron.io/apps/mastodon/#scaling
        https://docs.joinmastodon.org/admin/scaling/

        ? πŸ™‚

        I've not hit the need yet, but I think all you need to do is adjust the figures in /app/data/config.sh

        #!/bin/bash
        
        # Setup scaling related environment variables here - https://docs.joinmastodon.org/admin/scaling/
        
        # Puma
        export WEB_CONCURRENCY=2    # number of worker processes
        export MAX_THREADS=5        # the number of threads per process
        
        # Streaming API
        export STREAMING_CLUSTER_NUM=1  # number of worker processes
        
        # Sidekiq
        export SIDEKIQ_THREADS=2
        export DB_POOL=25               # must be at least the same as the number of threads
        

        I use Cloudron with Gandi & Hetzner

        doodlemania2 1 Reply Last reply Reply Quote 2
        • doodlemania2
          doodlemania2 App Dev @jdaviescoates last edited by

          @jdaviescoates ohhhh! doinks - thanks sir

          doodlemania2 1 Reply Last reply Reply Quote 1
          • doodlemania2
            doodlemania2 App Dev @doodlemania2 last edited by

            that did the trick nicely!

            A 1 Reply Last reply Reply Quote 2
            • A
              alwynispat @doodlemania2 last edited by

              @doodlemania2 what’s your setup now?
              here's mine:

              #!/bin/bash
              
              # Setup scaling related environment variables here - https://docs.joinmastodon.org/admin/scaling/
              
              # Puma
              export WEB_CONCURRENCY=3    # number of worker processes
              export MAX_THREADS=10        # the number of threads per process
              
              # Streaming API
              export STREAMING_CLUSTER_NUM=3  # number of worker processes
              export DB_POOL=15
              
              # Sidekiq
              export SIDEKIQ_THREADS=75
              export DB_POOL=75               # must be at least the same as the number of threads
              export MALLOC_ARENA_MAX=2
              
              
              

              Find me at Mastodon

              doodlemania2 1 Reply Last reply Reply Quote 1
              • doodlemania2
                doodlemania2 App Dev @alwynispat last edited by

                @alwynispat you've got DB_POOL specified twice πŸ™‚
                I'm doing:
                be945764-0726-41d0-8886-d387d17c123a-image.png

                1 Reply Last reply Reply Quote 2
                • Topic has been marked as a question  nebulon nebulon 
                • Topic has been marked as solved  nebulon nebulon 
                • First post
                  Last post
                Powered by NodeBB