Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Shaarli
  3. Shaarli app crashing constantly due to indexing bots (Yandex & co)

Shaarli app crashing constantly due to indexing bots (Yandex & co)

Scheduled Pinned Locked Moved Shaarli
6 Posts 3 Posters 333 Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • SansGuidonS Offline
    SansGuidonS Offline
    SansGuidon
    wrote on last edited by SansGuidon
    #1

    My Shaarli app on Cloudron has been crashing for the past hour, but it seems I'm unable to make use of iptables in the container nor configure apache rules to block those bots.
    I've of course blocked the user-agent via the Security config of Shaarli but it does not seem enough.
    How to mitigate this? I can't use Shaarli because of that or only for a couple seconds after restarting it. My Shaarli is a small instance.
    Workarounds exist https://sublimecoding.com/blocking-bots-in-apache-using-htaccess/ or https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/blob/master/Apache_2.4/README.md but require access to the configuration (apache/.htaccess...).

    Any idea? For now it's Shaarli, tomorrow it could be another web app on my instance, and I'm unsure to want to maintain my own block list and risk blocking myself by accident, even if that seem to be a viable solution? https://docs.cloudron.io/networking/#blocklist

    How do you mitigate this kind of spam?

    About me / Now

    1 Reply Last reply
    0
    • J joseph moved this topic from Support on
    • J Offline
      J Offline
      joseph
      Staff
      wrote on last edited by joseph
      #2

      If the bots are from fixed known IPs , then just block them using https://docs.cloudron.io/networking/#blocklist

      If the bots are all over, then this is like a DDoS. You will need to use some service like cloudflare to counteract this.

      edit: fixed link

      SansGuidonS 1 Reply Last reply
      0
      • J joseph

        If the bots are from fixed known IPs , then just block them using https://docs.cloudron.io/networking/#blocklist

        If the bots are all over, then this is like a DDoS. You will need to use some service like cloudflare to counteract this.

        edit: fixed link

        SansGuidonS Offline
        SansGuidonS Offline
        SansGuidon
        wrote on last edited by
        #3

        @joseph thanks! how to make use of the linked change for my shaarli instance? I'm not fully grasping how they relate.

        Regarding CF I'd prefer to avoid them, I'll never give them credit nor let their bloated modern firewall interfer with my stuff. They already ruin the web enough. So I prefer blocking by IP I guess.... 🤔🙏🏼

        About me / Now

        1 Reply Last reply
        2
        • fbartelsF Offline
          fbartelsF Offline
          fbartels
          App Dev
          wrote on last edited by
          #4

          Maybe he meant https://docs.cloudron.io/networking/#blocklist instead

          SansGuidonS 1 Reply Last reply
          2
          • fbartelsF fbartels

            Maybe he meant https://docs.cloudron.io/networking/#blocklist instead

            SansGuidonS Offline
            SansGuidonS Offline
            SansGuidon
            wrote on last edited by SansGuidon
            #5

            @fbartels in that case that's good as it's the option I've already tried as mentioned at the end of original post. It took me a while to find about it's existence though because it's the first time I had to face such an spam on my instance, usually blocking user agents was enough 😜

            About me / Now

            fbartelsF 1 Reply Last reply
            1
            • SansGuidonS SansGuidon

              @fbartels in that case that's good as it's the option I've already tried as mentioned at the end of original post. It took me a while to find about it's existence though because it's the first time I had to face such an spam on my instance, usually blocking user agents was enough 😜

              fbartelsF Offline
              fbartelsF Offline
              fbartels
              App Dev
              wrote on last edited by
              #6

              @SansGuidon I guess I should have completely reread your initial post before dropping the link 😅

              1 Reply Last reply
              2
              Reply
              • Reply as topic
              Log in to reply
              • Oldest to Newest
              • Newest to Oldest
              • Most Votes


              • Login

              • Don't have an account? Register

              • Login or register to search.
              • First post
                Last post
              0
              • Categories
              • Recent
              • Tags
              • Popular
              • Bookmarks
              • Search