Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

Cloudron Forum

Apps | Demo | Docs | Install

Run Additional Sidekiqs

Scheduled Pinned Locked Moved Solved Mastodon
6 Posts 3 Posters 150 Views
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • doodlemania2D Offline
    doodlemania2D Offline
    doodlemania2 App Dev
    wrote on last edited by girish
    #1

    I think I may have gotten to a threshold of sorts. The sidekiq running in my Mastodon instance isn't able to keep up. Top shows it running in the container no problem, but apparently I need more than one instance to get some parallelism rocking. Top also showed the container wasn't pegged at 100% CPU, so it had room to grow. I also bumped up the Resources from default 50% to 75%.

    So, I ended up with about a 6 hr backlog (around 40,000 messages) in my queues. In order to get them to clear, I opened up two terminal instances and manually ran bundle exec sidekiq in each and that chewed through them pretty quickly. Once the queues were drained, I closed them out but it started building back up again - the default sidekiq is definitely running, just a lot slower than needed.

    So, wondering if we could have a config option to ramp additional sidekiqs up?

    jdaviescoatesJ 1 Reply Last reply
    2
  • jdaviescoatesJ Offline
    jdaviescoatesJ Offline
    jdaviescoates
    replied to doodlemania2 on last edited by
    #2

    @doodlemania2 have you seen:

    https://docs.cloudron.io/apps/mastodon/#scaling
    https://docs.joinmastodon.org/admin/scaling/

    ? πŸ™‚

    I've not hit the need yet, but I think all you need to do is adjust the figures in /app/data/config.sh

    #!/bin/bash
    
    # Setup scaling related environment variables here - https://docs.joinmastodon.org/admin/scaling/
    
    # Puma
    export WEB_CONCURRENCY=2    # number of worker processes
    export MAX_THREADS=5        # the number of threads per process
    
    # Streaming API
    export STREAMING_CLUSTER_NUM=1  # number of worker processes
    
    # Sidekiq
    export SIDEKIQ_THREADS=2
    export DB_POOL=25               # must be at least the same as the number of threads
    

    I use Cloudron with Gandi & Hetzner

    doodlemania2D 1 Reply Last reply
    2
  • doodlemania2D Offline
    doodlemania2D Offline
    doodlemania2 App Dev
    replied to jdaviescoates on last edited by
    #3

    @jdaviescoates ohhhh! doinks - thanks sir

    doodlemania2D 1 Reply Last reply
    1
  • doodlemania2D Offline
    doodlemania2D Offline
    doodlemania2 App Dev
    replied to doodlemania2 on last edited by
    #4

    that did the trick nicely!

    A 1 Reply Last reply
    2
  • A Offline
    A Offline
    alwynispat
    replied to doodlemania2 on last edited by
    #5

    @doodlemania2 what’s your setup now?
    here's mine:

    #!/bin/bash
    
    # Setup scaling related environment variables here - https://docs.joinmastodon.org/admin/scaling/
    
    # Puma
    export WEB_CONCURRENCY=3    # number of worker processes
    export MAX_THREADS=10        # the number of threads per process
    
    # Streaming API
    export STREAMING_CLUSTER_NUM=3  # number of worker processes
    export DB_POOL=15
    
    # Sidekiq
    export SIDEKIQ_THREADS=75
    export DB_POOL=75               # must be at least the same as the number of threads
    export MALLOC_ARENA_MAX=2
    
    
    

    Find me at Mastodon

    doodlemania2D 1 Reply Last reply
    1
  • doodlemania2D Offline
    doodlemania2D Offline
    doodlemania2 App Dev
    replied to alwynispat on last edited by
    #6

    @alwynispat you've got DB_POOL specified twice πŸ™‚
    I'm doing:
    be945764-0726-41d0-8886-d387d17c123a-image.png

    1 Reply Last reply
    2
  • nebulonN nebulon marked this topic as a question on
  • nebulonN nebulon has marked this topic as solved on

  • Login

  • Don't have an account? Register

  • Login or register to search.
  • First post
    Last post
0
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Login

  • Don't have an account? Register

  • Login or register to search.