Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Discuss
  3. Crowdsec Install guide for cloudron purposes

Crowdsec Install guide for cloudron purposes

Scheduled Pinned Locked Moved Discuss
25 Posts 8 Posters 4.1k Views 11 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M Mastadamus

    Warning

    Please install at your own risk. Cloudron does not support installing extra applications and packages on the server.

    Install crowdsec, IPtables bouncer, and log4j detection collection on cloudron and reconfigure cloudron nginx conf for default logging.


    Install crowdsec

    sudo curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
    sudo apt install crowdsec
    

    (during the installation process Crowdsec install should install the appropriate "collections" which consists of parsers, and rules for the log sources on your cloudron.

    Edit nginx.conf file to put in default logging.

    • SSH into your cloudron
    • using text editor of your choice open the nginx.conf file found at /etc/nginx/nginx.conf
    • We want to changed the logging section to look like it does in this example nginx.conf file.
    user www-data;
    
    # detect based on available CPU cores
    worker_processes  auto;
    
    # this is 4096 by default. See /proc/<PID>/limits and /etc/security/limits.conf
    # usually twice the worker_connections (one for uptsream, one for downstream)
    # see also LimitNOFILE=16384 in systemd drop-in
    worker_rlimit_nofile 8192;
    
    pid /run/nginx.pid;
    
    events {
        # a single worker has these many simultaneous connections max
        worker_connections  4096;
    }
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
    
        # the collectd config depends on this log format
    
    
    
    
        # required for long host names
        server_names_hash_bucket_size 128;
    
        access_log /var/log/nginx/access.log combined;
    
        sendfile        on;
    
        # timeout for client to finish sending headers
        client_header_timeout 30s;
    
        # timeout for reading client request body (successive read timeout and not w                                                                             hole body!)
        client_body_timeout 60s;
    
        # keep-alive connections timeout in 65s. this is because many browsers timeo                                                                             ut in 60 seconds
        keepalive_timeout  65s;
    
        # zones for rate limiting
        limit_req_zone $binary_remote_addr zone=admin_login:10m rate=10r/s; # 10 req                                                                             uest a second
    
        include applications/*.conf;
    

    Install bouncer

    sudo apt install crowdsec-firewall-bouncer-iptables
    

    Install Log4j Collection

    sudo cscli hub update
    sudo cscli scenarios install crowdsecurity/apache_log4j2_cve-2021-44228
    sudo systemctl reload crowdsec
    

    Installation of crowdsec metabase docker dashboard

    1. to install the docker container on port 8181(may change this as desired. (Note this should be for internal network access only.) Do not open this up to the internet. It may be better to try to integrate this with the metabase app that comes with cloudron available in the appstore. I haven't dug into that yet though. )
    sudo cscli dashboard setup -l 0.0.0.0 -p 8181 --password < insert password>
    
    1. To make persistent.
    • Identify the crowdsec/metabase container ID number
    docker ps
    
    • persistant command
    sudo docker update --restart=unless-stopped <container ID number > 
    

    Access metabase

    1. you can access metabase by navigating to "http://yourIP:8181
    2. your credentials will be crowdsec@crowdsec.net and whatever you set the password.

    References:

    https://docs.crowdsec.net/docs/getting_started/install_crowdsec

    M Offline
    M Offline
    Mastadamus
    wrote on last edited by
    #2

    @mastadamus @girish do you know someone who might be able to help me figure out how to get the crowdsec metabase templates into the metabase app available to install from the cloudron store and connect them? IF we could make this happen, we would eliminate having to do this janky install of the crowdsec metabase docker container. Another possible alternative is to package up the crowdsec metabase image for cloudron? would that be possible?

    rmdesR 1 Reply Last reply
    2
    • M Mastadamus

      @mastadamus @girish do you know someone who might be able to help me figure out how to get the crowdsec metabase templates into the metabase app available to install from the cloudron store and connect them? IF we could make this happen, we would eliminate having to do this janky install of the crowdsec metabase docker container. Another possible alternative is to package up the crowdsec metabase image for cloudron? would that be possible?

      rmdesR Offline
      rmdesR Offline
      rmdes
      wrote on last edited by
      #3

      @mastadamus I there should be a way to do this, because crowdsec themselves custom build a metabase image, to include the db and other pre implemented stuff.
      look here : https://github.com/crowdsecurity/example-docker-compose
      check out the "dashboard" folder, there is a Dockerfile
      they start from the metabase image
      and then just do this : RUN mkdir /data/ && wget https://crowdsec-statics-assets.s3-eu-west-1.amazonaws.com/metabase_sqlite.zip && unzip metabase_sqlite.zip -d /data/

      I think this sqlite file is what makes metabase ready for crowdsec data from the get go ?

      1 Reply Last reply
      0
      • M Mastadamus

        Warning

        Please install at your own risk. Cloudron does not support installing extra applications and packages on the server.

        Install crowdsec, IPtables bouncer, and log4j detection collection on cloudron and reconfigure cloudron nginx conf for default logging.


        Install crowdsec

        sudo curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
        sudo apt install crowdsec
        

        (during the installation process Crowdsec install should install the appropriate "collections" which consists of parsers, and rules for the log sources on your cloudron.

        Edit nginx.conf file to put in default logging.

        • SSH into your cloudron
        • using text editor of your choice open the nginx.conf file found at /etc/nginx/nginx.conf
        • We want to changed the logging section to look like it does in this example nginx.conf file.
        user www-data;
        
        # detect based on available CPU cores
        worker_processes  auto;
        
        # this is 4096 by default. See /proc/<PID>/limits and /etc/security/limits.conf
        # usually twice the worker_connections (one for uptsream, one for downstream)
        # see also LimitNOFILE=16384 in systemd drop-in
        worker_rlimit_nofile 8192;
        
        pid /run/nginx.pid;
        
        events {
            # a single worker has these many simultaneous connections max
            worker_connections  4096;
        }
        
        http {
            include       mime.types;
            default_type  application/octet-stream;
        
            # the collectd config depends on this log format
        
        
        
        
            # required for long host names
            server_names_hash_bucket_size 128;
        
            access_log /var/log/nginx/access.log combined;
        
            sendfile        on;
        
            # timeout for client to finish sending headers
            client_header_timeout 30s;
        
            # timeout for reading client request body (successive read timeout and not w                                                                             hole body!)
            client_body_timeout 60s;
        
            # keep-alive connections timeout in 65s. this is because many browsers timeo                                                                             ut in 60 seconds
            keepalive_timeout  65s;
        
            # zones for rate limiting
            limit_req_zone $binary_remote_addr zone=admin_login:10m rate=10r/s; # 10 req                                                                             uest a second
        
            include applications/*.conf;
        

        Install bouncer

        sudo apt install crowdsec-firewall-bouncer-iptables
        

        Install Log4j Collection

        sudo cscli hub update
        sudo cscli scenarios install crowdsecurity/apache_log4j2_cve-2021-44228
        sudo systemctl reload crowdsec
        

        Installation of crowdsec metabase docker dashboard

        1. to install the docker container on port 8181(may change this as desired. (Note this should be for internal network access only.) Do not open this up to the internet. It may be better to try to integrate this with the metabase app that comes with cloudron available in the appstore. I haven't dug into that yet though. )
        sudo cscli dashboard setup -l 0.0.0.0 -p 8181 --password < insert password>
        
        1. To make persistent.
        • Identify the crowdsec/metabase container ID number
        docker ps
        
        • persistant command
        sudo docker update --restart=unless-stopped <container ID number > 
        

        Access metabase

        1. you can access metabase by navigating to "http://yourIP:8181
        2. your credentials will be crowdsec@crowdsec.net and whatever you set the password.

        References:

        https://docs.crowdsec.net/docs/getting_started/install_crowdsec

        rmdesR Offline
        rmdesR Offline
        rmdes
        wrote on last edited by
        #4

        @mastadamus

        I had to specify a port for crowdsec service to be able to start, after trying a few ones that were already being used I settled for 9999
        other than that, crowdsec is running just fine on my cloudron server

        rmdesR 1 Reply Last reply
        1
        • rmdesR rmdes

          @mastadamus

          I had to specify a port for crowdsec service to be able to start, after trying a few ones that were already being used I settled for 9999
          other than that, crowdsec is running just fine on my cloudron server

          rmdesR Offline
          rmdesR Offline
          rmdes
          wrote on last edited by rmdes
          #5

          my nginx bouncer manually installed (not from packages) was easy to test along in a cloudron context but, do NOT apt install crowdsec-firewall-bouncer-iptables or it will mess up with your nginx and cloudron beyond mess.

          Latest version does not mess anymore and beside using a free docker port, it seems crowdsec and cloudron can run fine.

          M rmdesR 2 Replies Last reply
          1
          • rmdesR rmdes

            my nginx bouncer manually installed (not from packages) was easy to test along in a cloudron context but, do NOT apt install crowdsec-firewall-bouncer-iptables or it will mess up with your nginx and cloudron beyond mess.

            Latest version does not mess anymore and beside using a free docker port, it seems crowdsec and cloudron can run fine.

            M Offline
            M Offline
            Mastadamus
            wrote on last edited by
            #6

            @rmdes I installed iptables bouncer and its working great. The nginx bouncer messed up my nginx service though and caused it not to start. I was not able to get nginx bouncer working.

            rmdesR 1 Reply Last reply
            0
            • K Offline
              K Offline
              klausagnoletti
              wrote on last edited by
              #7

              Brilliant work! I am head of community at CrowdSec and a good place to talk about a CrowdSec integration to Cloudron and more official support would be at our new
              Discord server. See you there!

              1 Reply Last reply
              1
              • M Mastadamus

                @rmdes I installed iptables bouncer and its working great. The nginx bouncer messed up my nginx service though and caused it not to start. I was not able to get nginx bouncer working.

                rmdesR Offline
                rmdesR Offline
                rmdes
                wrote on last edited by
                #8

                @mastadamus this is logical in fact, this is because the firewall bouncer intercept the traffic at the iptable level and stop 99.% of scenarios before traffic even hit the nginx process.

                So if you really want to test the nginx bouncer you need to disable the firewall bouncer

                M 1 Reply Last reply
                1
                • rmdesR rmdes

                  my nginx bouncer manually installed (not from packages) was easy to test along in a cloudron context but, do NOT apt install crowdsec-firewall-bouncer-iptables or it will mess up with your nginx and cloudron beyond mess.

                  Latest version does not mess anymore and beside using a free docker port, it seems crowdsec and cloudron can run fine.

                  rmdesR Offline
                  rmdesR Offline
                  rmdes
                  wrote on last edited by rmdes
                  #9

                  After more testing this is my report and steps to have it running smooth:

                  • adding crowdsec repo
                  • apt install crowdsec
                  • apt install crowdsec-firewall-bouncer-iptables

                  it should wizard the first init properly and add API keys accordingly, sometimes it seems the 2 services starts before the correct state of the whole crowdsec is correctly setup, so the solution is to (stop both services) make sure the list of follow up to the manual setup are correctly setup, the cscli tool has list, status and other commands to check if everything is fine.


                  I had to tweak crowdsec-firewall-bouncer.yml to this
                  97ffc412-a998-419d-9716-12af3ca676ed-image.png

                  Docker

                  I used this approach to get all the containers (acquis.yml)

                  # Docker context
                  source : docker
                  container_name_regexp:
                    - ^[a-zA-Z0-9_.-]*$
                  labels:
                    type: log_type
                  ---
                  

                  I used port 9999 in config.yml and accordingly into
                  local_api_credentials.yaml
                  bouncers/crowdsec-firewall-bouncer.yaml
                  bouncers/crowdsec-nginx-bouncer.conf

                  If you regenerate a bouncer API you need to make sure the new API is in the bouncer/yml file accordingly.

                  when everything is fine,
                  d627b6e4-1dc4-4593-b182-e7030791c82b-image.png

                  If you see errors in the crowdsec log, it's probably either machine not validated, wrong API number in lapi = crowdsec firewall service not running (main reason why often)

                  cscli machine list IP address need to be assigned, ortherwise it means the machine is not being used or there is an API issue.
                  ecc89412-d9d7-4681-aedd-d336f2ba27f6-image.png

                  cscli lapi status (success)
                  cscli capi status (success)
                  cscli metrics (return data)

                  install linux/ssh/nginx/docker scenarios using cscli
                  sudo systemctl enable/start/status crowdsec-firewall-bouncer.service crowdsec.service

                  need to be healthy active/loaded at all time


                  Metabase

                  to handle the metabase feature, I simply run it this way:
                  cscli dashboard setup -l 127.0.0.1 -p 12345 --password YourPassw0rd
                  keep provided credentials (you can change them when logged)

                  and then access the frontend first doing on a new terminal
                  ssh -L 3000:127.0.0.1:12345 user@host

                  and in the browser : http://localhost:3000
                  5245a045-8b9f-4db1-b891-1093b15e41fa-image.png
                  16ac8802-a765-4810-b2ba-9ef9accd3d66-image.png

                  718757fe-bf42-47c5-94a1-75f16508341e-image.png

                  This leaves metabase isolated from the internet and only accessible via SSH

                  No Metabase

                  finally if you don't want to mess with having an "outside" cloudron metabase running to have a dashboard locally, just don't use cscli dashboard, if you want to remove it you can just cscli dashboard stop or remove

                  and then use cscli enroll command to have the crowdsec provided "as a service" dashboard https://app.crowdsec.net

                  e3a950d1-b2e0-488f-a561-e92d95062973-image.png

                  I have 1 machine inactive in this setup because it is the one active on the nginx bouncer that is currently in place but not being used.

                  rmdesR BrutalBirdieB 2 Replies Last reply
                  2
                  • rmdesR rmdes

                    @mastadamus this is logical in fact, this is because the firewall bouncer intercept the traffic at the iptable level and stop 99.% of scenarios before traffic even hit the nginx process.

                    So if you really want to test the nginx bouncer you need to disable the firewall bouncer

                    M Offline
                    M Offline
                    Mastadamus
                    wrote on last edited by
                    #10

                    @rmdes I tried nginx bouncer BEFORE installing iptables bouncer and it crashed my nginx and wouldn't let me bring it up. I then reverted to previous image and installed iptables bouncer which works great.

                    1 Reply Last reply
                    1
                    • rmdesR rmdes

                      After more testing this is my report and steps to have it running smooth:

                      • adding crowdsec repo
                      • apt install crowdsec
                      • apt install crowdsec-firewall-bouncer-iptables

                      it should wizard the first init properly and add API keys accordingly, sometimes it seems the 2 services starts before the correct state of the whole crowdsec is correctly setup, so the solution is to (stop both services) make sure the list of follow up to the manual setup are correctly setup, the cscli tool has list, status and other commands to check if everything is fine.


                      I had to tweak crowdsec-firewall-bouncer.yml to this
                      97ffc412-a998-419d-9716-12af3ca676ed-image.png

                      Docker

                      I used this approach to get all the containers (acquis.yml)

                      # Docker context
                      source : docker
                      container_name_regexp:
                        - ^[a-zA-Z0-9_.-]*$
                      labels:
                        type: log_type
                      ---
                      

                      I used port 9999 in config.yml and accordingly into
                      local_api_credentials.yaml
                      bouncers/crowdsec-firewall-bouncer.yaml
                      bouncers/crowdsec-nginx-bouncer.conf

                      If you regenerate a bouncer API you need to make sure the new API is in the bouncer/yml file accordingly.

                      when everything is fine,
                      d627b6e4-1dc4-4593-b182-e7030791c82b-image.png

                      If you see errors in the crowdsec log, it's probably either machine not validated, wrong API number in lapi = crowdsec firewall service not running (main reason why often)

                      cscli machine list IP address need to be assigned, ortherwise it means the machine is not being used or there is an API issue.
                      ecc89412-d9d7-4681-aedd-d336f2ba27f6-image.png

                      cscli lapi status (success)
                      cscli capi status (success)
                      cscli metrics (return data)

                      install linux/ssh/nginx/docker scenarios using cscli
                      sudo systemctl enable/start/status crowdsec-firewall-bouncer.service crowdsec.service

                      need to be healthy active/loaded at all time


                      Metabase

                      to handle the metabase feature, I simply run it this way:
                      cscli dashboard setup -l 127.0.0.1 -p 12345 --password YourPassw0rd
                      keep provided credentials (you can change them when logged)

                      and then access the frontend first doing on a new terminal
                      ssh -L 3000:127.0.0.1:12345 user@host

                      and in the browser : http://localhost:3000
                      5245a045-8b9f-4db1-b891-1093b15e41fa-image.png
                      16ac8802-a765-4810-b2ba-9ef9accd3d66-image.png

                      718757fe-bf42-47c5-94a1-75f16508341e-image.png

                      This leaves metabase isolated from the internet and only accessible via SSH

                      No Metabase

                      finally if you don't want to mess with having an "outside" cloudron metabase running to have a dashboard locally, just don't use cscli dashboard, if you want to remove it you can just cscli dashboard stop or remove

                      and then use cscli enroll command to have the crowdsec provided "as a service" dashboard https://app.crowdsec.net

                      e3a950d1-b2e0-488f-a561-e92d95062973-image.png

                      I have 1 machine inactive in this setup because it is the one active on the nginx bouncer that is currently in place but not being used.

                      rmdesR Offline
                      rmdesR Offline
                      rmdes
                      wrote on last edited by
                      #11

                      the main consequences I have seen is a drastic drop in all the wordpress (previously using wordfence or else) attacks/prob/vuln scan etc..after a week without these plugins doing anything (the firewall intercept these scenarios before they reach nginx) my wordfence logs are now bots/humans/myself, no more attacks.

                      it's just brutal since Armada host quite a few domains running wordpress, so basically the surface to attack drop's considerably with crowdsec agent monitoring the door, the Wordpress containers don't even get the bots usually hitting the domains, so I also have seen a drop in CPU/Mem usage overall everywhere, it's now running since 2 weeks without any problem.

                      I'm sure now that the first issue I had was because of the manual installation at that point in time, for some reason it pulled another version of nginx, but even if you reproduce these steps now, the installation just work fines and if you mess with the agent API, it's really easy to replace the API accordingly, have all the pieces of the YML files correctly configured, make sure systemd service are enabled and started and running and voila !

                      If you want to go the safe way, just use the Firewall (in fact if you use it, the nginx bouncer is irrelevant) in my experience setting up metabase the way I did is safe and does not impact cloudron in any way that I could see.

                      jdaviescoatesJ 1 Reply Last reply
                      3
                      • rmdesR rmdes

                        the main consequences I have seen is a drastic drop in all the wordpress (previously using wordfence or else) attacks/prob/vuln scan etc..after a week without these plugins doing anything (the firewall intercept these scenarios before they reach nginx) my wordfence logs are now bots/humans/myself, no more attacks.

                        it's just brutal since Armada host quite a few domains running wordpress, so basically the surface to attack drop's considerably with crowdsec agent monitoring the door, the Wordpress containers don't even get the bots usually hitting the domains, so I also have seen a drop in CPU/Mem usage overall everywhere, it's now running since 2 weeks without any problem.

                        I'm sure now that the first issue I had was because of the manual installation at that point in time, for some reason it pulled another version of nginx, but even if you reproduce these steps now, the installation just work fines and if you mess with the agent API, it's really easy to replace the API accordingly, have all the pieces of the YML files correctly configured, make sure systemd service are enabled and started and running and voila !

                        If you want to go the safe way, just use the Firewall (in fact if you use it, the nginx bouncer is irrelevant) in my experience setting up metabase the way I did is safe and does not impact cloudron in any way that I could see.

                        jdaviescoatesJ Offline
                        jdaviescoatesJ Offline
                        jdaviescoates
                        wrote on last edited by
                        #12

                        @rmdes that all sounds great. I wonder if Crowdsec could be baked into Cloudron itself?

                        I use Cloudron with Gandi & Hetzner

                        rmdesR 1 Reply Last reply
                        1
                        • jdaviescoatesJ jdaviescoates

                          @rmdes that all sounds great. I wonder if Crowdsec could be baked into Cloudron itself?

                          rmdesR Offline
                          rmdesR Offline
                          rmdes
                          wrote on last edited by
                          #13

                          @jdaviescoates I know the crowdsec people are working "upstream" with all kinds of actors to get this done for many hosting/implementations out there, in my experience, since both Cloudron and Crowdsec use iptables this integration is already the case, what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already :

                          • has a Docker type
                          • IPtables
                          • Can plug on any SystemD service to parse logs

                          what could probably be optimized is how crowdsec "watches" all the docker containers and perhaps make sure the implementation does not disrupt cloudron operations in any way, make it bullet proof install tested.

                          K jdaviescoatesJ 2 Replies Last reply
                          2
                          • rmdesR rmdes

                            @jdaviescoates I know the crowdsec people are working "upstream" with all kinds of actors to get this done for many hosting/implementations out there, in my experience, since both Cloudron and Crowdsec use iptables this integration is already the case, what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already :

                            • has a Docker type
                            • IPtables
                            • Can plug on any SystemD service to parse logs

                            what could probably be optimized is how crowdsec "watches" all the docker containers and perhaps make sure the implementation does not disrupt cloudron operations in any way, make it bullet proof install tested.

                            K Offline
                            K Offline
                            klausagnoletti
                            wrote on last edited by
                            #14

                            @rmdes I am head of community at CrowdSec and we would be happy to work with anyone from the Cloudron community on this. For this purpose it would be a great idea to join our Discord community and talk about it there 🙂

                            rmdesR 1 Reply Last reply
                            2
                            • rmdesR rmdes

                              @jdaviescoates I know the crowdsec people are working "upstream" with all kinds of actors to get this done for many hosting/implementations out there, in my experience, since both Cloudron and Crowdsec use iptables this integration is already the case, what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already :

                              • has a Docker type
                              • IPtables
                              • Can plug on any SystemD service to parse logs

                              what could probably be optimized is how crowdsec "watches" all the docker containers and perhaps make sure the implementation does not disrupt cloudron operations in any way, make it bullet proof install tested.

                              jdaviescoatesJ Offline
                              jdaviescoatesJ Offline
                              jdaviescoates
                              wrote on last edited by
                              #15

                              @rmdes said in Crowdsec Install guide for cloudron purposes:

                              what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already

                              That would have massive added value to me.

                              I wouldn't trust myself to not mess something up doing it manually!

                              And anything that makes life easier and simpler is welcome. So just having some option in the Cloudron admin that could enable Crowdsec would be ideal.

                              I use Cloudron with Gandi & Hetzner

                              1 Reply Last reply
                              1
                              • K klausagnoletti

                                @rmdes I am head of community at CrowdSec and we would be happy to work with anyone from the Cloudron community on this. For this purpose it would be a great idea to join our Discord community and talk about it there 🙂

                                rmdesR Offline
                                rmdesR Offline
                                rmdes
                                wrote on last edited by
                                #16

                                @klausagnoletti I'm already there 🙂 but not a Cloudron dev or anything, but yeah it would be amazing the have the existing metabase cloudron app in "flavors" one could be the default app we have now, another one could bootstrap crowdsec db and be a one click install for the underlying crowdsec installed in the host but I have no idea if it's possible.

                                1 Reply Last reply
                                3
                                • necrevistonnezrN necrevistonnezr referenced this topic on
                                • M Mastadamus

                                  Warning

                                  Please install at your own risk. Cloudron does not support installing extra applications and packages on the server.

                                  Install crowdsec, IPtables bouncer, and log4j detection collection on cloudron and reconfigure cloudron nginx conf for default logging.


                                  Install crowdsec

                                  sudo curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
                                  sudo apt install crowdsec
                                  

                                  (during the installation process Crowdsec install should install the appropriate "collections" which consists of parsers, and rules for the log sources on your cloudron.

                                  Edit nginx.conf file to put in default logging.

                                  • SSH into your cloudron
                                  • using text editor of your choice open the nginx.conf file found at /etc/nginx/nginx.conf
                                  • We want to changed the logging section to look like it does in this example nginx.conf file.
                                  user www-data;
                                  
                                  # detect based on available CPU cores
                                  worker_processes  auto;
                                  
                                  # this is 4096 by default. See /proc/<PID>/limits and /etc/security/limits.conf
                                  # usually twice the worker_connections (one for uptsream, one for downstream)
                                  # see also LimitNOFILE=16384 in systemd drop-in
                                  worker_rlimit_nofile 8192;
                                  
                                  pid /run/nginx.pid;
                                  
                                  events {
                                      # a single worker has these many simultaneous connections max
                                      worker_connections  4096;
                                  }
                                  
                                  http {
                                      include       mime.types;
                                      default_type  application/octet-stream;
                                  
                                      # the collectd config depends on this log format
                                  
                                  
                                  
                                  
                                      # required for long host names
                                      server_names_hash_bucket_size 128;
                                  
                                      access_log /var/log/nginx/access.log combined;
                                  
                                      sendfile        on;
                                  
                                      # timeout for client to finish sending headers
                                      client_header_timeout 30s;
                                  
                                      # timeout for reading client request body (successive read timeout and not w                                                                             hole body!)
                                      client_body_timeout 60s;
                                  
                                      # keep-alive connections timeout in 65s. this is because many browsers timeo                                                                             ut in 60 seconds
                                      keepalive_timeout  65s;
                                  
                                      # zones for rate limiting
                                      limit_req_zone $binary_remote_addr zone=admin_login:10m rate=10r/s; # 10 req                                                                             uest a second
                                  
                                      include applications/*.conf;
                                  

                                  Install bouncer

                                  sudo apt install crowdsec-firewall-bouncer-iptables
                                  

                                  Install Log4j Collection

                                  sudo cscli hub update
                                  sudo cscli scenarios install crowdsecurity/apache_log4j2_cve-2021-44228
                                  sudo systemctl reload crowdsec
                                  

                                  Installation of crowdsec metabase docker dashboard

                                  1. to install the docker container on port 8181(may change this as desired. (Note this should be for internal network access only.) Do not open this up to the internet. It may be better to try to integrate this with the metabase app that comes with cloudron available in the appstore. I haven't dug into that yet though. )
                                  sudo cscli dashboard setup -l 0.0.0.0 -p 8181 --password < insert password>
                                  
                                  1. To make persistent.
                                  • Identify the crowdsec/metabase container ID number
                                  docker ps
                                  
                                  • persistant command
                                  sudo docker update --restart=unless-stopped <container ID number > 
                                  

                                  Access metabase

                                  1. you can access metabase by navigating to "http://yourIP:8181
                                  2. your credentials will be crowdsec@crowdsec.net and whatever you set the password.

                                  References:

                                  https://docs.crowdsec.net/docs/getting_started/install_crowdsec

                                  BrutalBirdieB Offline
                                  BrutalBirdieB Offline
                                  BrutalBirdie
                                  Partner
                                  wrote on last edited by BrutalBirdie
                                  #17

                                  I am now checking out CrowdSec as well.

                                  As of writing this the installation instruction from @Mastadamus where almost correct.

                                  Edit nginx.conf file to put in default logging

                                  Not needed, also the posted config is bad since it's missing the final closing curly bracket }

                                  ❎


                                  Everything else worked as described.


                                  Now to the bouncer.

                                  @rmdes said in Crowdsec Install guide for cloudron purposes:

                                  So if you really want to test the nginx bouncer you need to disable the firewall bouncer

                                  Ahhh ok, nah I will stick with the iptable bouncer.

                                  Like my work? Consider donating a drink. Cheers!

                                  1 Reply Last reply
                                  0
                                  • rmdesR rmdes

                                    After more testing this is my report and steps to have it running smooth:

                                    • adding crowdsec repo
                                    • apt install crowdsec
                                    • apt install crowdsec-firewall-bouncer-iptables

                                    it should wizard the first init properly and add API keys accordingly, sometimes it seems the 2 services starts before the correct state of the whole crowdsec is correctly setup, so the solution is to (stop both services) make sure the list of follow up to the manual setup are correctly setup, the cscli tool has list, status and other commands to check if everything is fine.


                                    I had to tweak crowdsec-firewall-bouncer.yml to this
                                    97ffc412-a998-419d-9716-12af3ca676ed-image.png

                                    Docker

                                    I used this approach to get all the containers (acquis.yml)

                                    # Docker context
                                    source : docker
                                    container_name_regexp:
                                      - ^[a-zA-Z0-9_.-]*$
                                    labels:
                                      type: log_type
                                    ---
                                    

                                    I used port 9999 in config.yml and accordingly into
                                    local_api_credentials.yaml
                                    bouncers/crowdsec-firewall-bouncer.yaml
                                    bouncers/crowdsec-nginx-bouncer.conf

                                    If you regenerate a bouncer API you need to make sure the new API is in the bouncer/yml file accordingly.

                                    when everything is fine,
                                    d627b6e4-1dc4-4593-b182-e7030791c82b-image.png

                                    If you see errors in the crowdsec log, it's probably either machine not validated, wrong API number in lapi = crowdsec firewall service not running (main reason why often)

                                    cscli machine list IP address need to be assigned, ortherwise it means the machine is not being used or there is an API issue.
                                    ecc89412-d9d7-4681-aedd-d336f2ba27f6-image.png

                                    cscli lapi status (success)
                                    cscli capi status (success)
                                    cscli metrics (return data)

                                    install linux/ssh/nginx/docker scenarios using cscli
                                    sudo systemctl enable/start/status crowdsec-firewall-bouncer.service crowdsec.service

                                    need to be healthy active/loaded at all time


                                    Metabase

                                    to handle the metabase feature, I simply run it this way:
                                    cscli dashboard setup -l 127.0.0.1 -p 12345 --password YourPassw0rd
                                    keep provided credentials (you can change them when logged)

                                    and then access the frontend first doing on a new terminal
                                    ssh -L 3000:127.0.0.1:12345 user@host

                                    and in the browser : http://localhost:3000
                                    5245a045-8b9f-4db1-b891-1093b15e41fa-image.png
                                    16ac8802-a765-4810-b2ba-9ef9accd3d66-image.png

                                    718757fe-bf42-47c5-94a1-75f16508341e-image.png

                                    This leaves metabase isolated from the internet and only accessible via SSH

                                    No Metabase

                                    finally if you don't want to mess with having an "outside" cloudron metabase running to have a dashboard locally, just don't use cscli dashboard, if you want to remove it you can just cscli dashboard stop or remove

                                    and then use cscli enroll command to have the crowdsec provided "as a service" dashboard https://app.crowdsec.net

                                    e3a950d1-b2e0-488f-a561-e92d95062973-image.png

                                    I have 1 machine inactive in this setup because it is the one active on the nginx bouncer that is currently in place but not being used.

                                    BrutalBirdieB Offline
                                    BrutalBirdieB Offline
                                    BrutalBirdie
                                    Partner
                                    wrote on last edited by
                                    #18

                                    @rmdes said in Crowdsec Install guide for cloudron purposes:

                                    I had to tweak crowdsec-firewall-bouncer.yml to this

                                    Where? What?

                                    https://docs.crowdsec.net/docs/bouncers/firewall/#configuration
                                    /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

                                    Ok.

                                    @rmdes said in Crowdsec Install guide for cloudron purposes:

                                    I used this approach to get all the containers (acquis.yml)

                                    Where? What?

                                    https://docs.crowdsec.net/docs/v1.0/acquisitions/format/#acquisition-directives
                                    /etc/crowdsec/acquis.yaml

                                    Ok.

                                    @rmdes said in Crowdsec Install guide for cloudron purposes:

                                    I used port 9999 in config.yml

                                    Where? What?

                                    https://docs.crowdsec.net/docs/configuration/crowdsec_configuration#configuration-directives
                                    /etc/crowdsec/config.yaml

                                    In yaml querry / json qurry format:
                                    yq '.api.server.listen_uri' < /etc/crowdsec/config.yaml
                                    127.0.0.1:8080 this changed to 127.0.0.1:9999 I assume.
                                    Changed to :9999

                                    @rmdes said in Crowdsec Install guide for cloudron purposes:

                                    and accordingly into
                                    local_api_credentials.yaml

                                    Where? What?

                                    https://docs.crowdsec.net/docs/next/local_api/intro/
                                    /etc/crowdsec/local_api_credentials.yaml

                                    In yaml querry langauge:
                                    yq '.url' < /etc/crowdsec/local_api_credentials.yaml
                                    http://127.0.0.1:8080 changed to :9999

                                    @rmdes said in Crowdsec Install guide for cloudron purposes:

                                    bouncers/crowdsec-firewall-bouncer.yaml

                                    Where? What?
                                    https://docs.crowdsec.net/docs/bouncers/firewall/
                                    /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

                                    yaml querry:
                                    yq '.api_url' < /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
                                    http://127.0.0.1:8080 changed to :9999

                                    @rmdes said in Crowdsec Install guide for cloudron purposes:

                                    install linux/ssh/nginx/docker scenarios using cscli

                                    What? How?

                                    https://docs.crowdsec.net/docs/v1.1/cscli/cscli_scenarios_install/
                                    cscli scenarios install
                                    Which scenarios can I install?
                                    https://hub.crowdsec.net/browse/

                                    Following your advice:

                                    cscli scenarios install linux
                                    cscli scenarios install sshd
                                    cscli scenarios install nginx
                                    

                                    I did not find and scenario for docker? @rmdes and now when I try cscli scenarios install linux I get an error.
                                    Well it seems this are not scenarios but collections.

                                    So it's https://docs.crowdsec.net/docs/v1.2/cscli/cscli_collections_install/

                                    cscli collections install crowdsecurity/linux
                                    

                                    Which is already installed by default, which also included sshd by default.

                                    I could not find docker at all.
                                    Maybe you can help out here @rmdes

                                    Like my work? Consider donating a drink. Cheers!

                                    BrutalBirdieB 1 Reply Last reply
                                    2
                                    • BrutalBirdieB BrutalBirdie

                                      @rmdes said in Crowdsec Install guide for cloudron purposes:

                                      I had to tweak crowdsec-firewall-bouncer.yml to this

                                      Where? What?

                                      https://docs.crowdsec.net/docs/bouncers/firewall/#configuration
                                      /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

                                      Ok.

                                      @rmdes said in Crowdsec Install guide for cloudron purposes:

                                      I used this approach to get all the containers (acquis.yml)

                                      Where? What?

                                      https://docs.crowdsec.net/docs/v1.0/acquisitions/format/#acquisition-directives
                                      /etc/crowdsec/acquis.yaml

                                      Ok.

                                      @rmdes said in Crowdsec Install guide for cloudron purposes:

                                      I used port 9999 in config.yml

                                      Where? What?

                                      https://docs.crowdsec.net/docs/configuration/crowdsec_configuration#configuration-directives
                                      /etc/crowdsec/config.yaml

                                      In yaml querry / json qurry format:
                                      yq '.api.server.listen_uri' < /etc/crowdsec/config.yaml
                                      127.0.0.1:8080 this changed to 127.0.0.1:9999 I assume.
                                      Changed to :9999

                                      @rmdes said in Crowdsec Install guide for cloudron purposes:

                                      and accordingly into
                                      local_api_credentials.yaml

                                      Where? What?

                                      https://docs.crowdsec.net/docs/next/local_api/intro/
                                      /etc/crowdsec/local_api_credentials.yaml

                                      In yaml querry langauge:
                                      yq '.url' < /etc/crowdsec/local_api_credentials.yaml
                                      http://127.0.0.1:8080 changed to :9999

                                      @rmdes said in Crowdsec Install guide for cloudron purposes:

                                      bouncers/crowdsec-firewall-bouncer.yaml

                                      Where? What?
                                      https://docs.crowdsec.net/docs/bouncers/firewall/
                                      /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

                                      yaml querry:
                                      yq '.api_url' < /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
                                      http://127.0.0.1:8080 changed to :9999

                                      @rmdes said in Crowdsec Install guide for cloudron purposes:

                                      install linux/ssh/nginx/docker scenarios using cscli

                                      What? How?

                                      https://docs.crowdsec.net/docs/v1.1/cscli/cscli_scenarios_install/
                                      cscli scenarios install
                                      Which scenarios can I install?
                                      https://hub.crowdsec.net/browse/

                                      Following your advice:

                                      cscli scenarios install linux
                                      cscli scenarios install sshd
                                      cscli scenarios install nginx
                                      

                                      I did not find and scenario for docker? @rmdes and now when I try cscli scenarios install linux I get an error.
                                      Well it seems this are not scenarios but collections.

                                      So it's https://docs.crowdsec.net/docs/v1.2/cscli/cscli_collections_install/

                                      cscli collections install crowdsecurity/linux
                                      

                                      Which is already installed by default, which also included sshd by default.

                                      I could not find docker at all.
                                      Maybe you can help out here @rmdes

                                      BrutalBirdieB Offline
                                      BrutalBirdieB Offline
                                      BrutalBirdie
                                      Partner
                                      wrote on last edited by
                                      #19

                                      Now I am running crowdsec on my cloudron.
                                      I will report back for more findings.

                                      Like my work? Consider donating a drink. Cheers!

                                      necrevistonnezrN rmdesR 2 Replies Last reply
                                      4
                                      • BrutalBirdieB BrutalBirdie

                                        Now I am running crowdsec on my cloudron.
                                        I will report back for more findings.

                                        necrevistonnezrN Offline
                                        necrevistonnezrN Offline
                                        necrevistonnezr
                                        wrote on last edited by
                                        #20

                                        @BrutalBirdie Thanks! It would be great if your, @rmdes and @Mastadamus findings could end up in a Cloudron doc in the end...!

                                        1 Reply Last reply
                                        3
                                        • BrutalBirdieB BrutalBirdie

                                          Now I am running crowdsec on my cloudron.
                                          I will report back for more findings.

                                          rmdesR Offline
                                          rmdesR Offline
                                          rmdes
                                          wrote on last edited by
                                          #21

                                          @BrutalBirdie Sorry have been busy and thanks for documenting my steps 👍 👍

                                          The docker part is done on the /etc/crowdsec/acquis.yml
                                          I'm not entirely convinced my regex call works appropriately
                                          or even that in the cloudron context this is entirely needed (I have one cloudron with & without not seeing much difference)

                                          # Docker context
                                          source : docker
                                          container_name_regexp:
                                            - ^[a-zA-Z0-9_.-]*$
                                          labels:
                                            type: log_type
                                          ---
                                          
                                          
                                          1 Reply Last reply
                                          3
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Bookmarks
                                          • Search