Crowdsec Install guide for cloudron purposes
-
Brilliant work! I am head of community at CrowdSec and a good place to talk about a CrowdSec integration to Cloudron and more official support would be at our new
Discord server. See you there! -
@rmdes I installed iptables bouncer and its working great. The nginx bouncer messed up my nginx service though and caused it not to start. I was not able to get nginx bouncer working.
@mastadamus this is logical in fact, this is because the firewall bouncer intercept the traffic at the iptable level and stop 99.% of scenarios before traffic even hit the nginx process.
So if you really want to test the nginx bouncer you need to disable the firewall bouncer
-
my nginx bouncer manually installed (not from packages) was easy to test along in a cloudron context but, do NOT apt install crowdsec-firewall-bouncer-iptables or it will mess up with your nginx and cloudron beyond mess.Latest version does not mess anymore and beside using a free docker port, it seems crowdsec and cloudron can run fine.
After more testing this is my report and steps to have it running smooth:
- adding crowdsec repo
- apt install crowdsec
- apt install crowdsec-firewall-bouncer-iptables
it should wizard the first init properly and add API keys accordingly, sometimes it seems the 2 services starts before the correct state of the whole crowdsec is correctly setup, so the solution is to (stop both services) make sure the list of follow up to the manual setup are correctly setup, the cscli tool has list, status and other commands to check if everything is fine.
I had to tweak crowdsec-firewall-bouncer.yml to this
Docker
I used this approach to get all the containers (acquis.yml)
# Docker context source : docker container_name_regexp: - ^[a-zA-Z0-9_.-]*$ labels: type: log_type ---I used port 9999 in config.yml and accordingly into
local_api_credentials.yaml
bouncers/crowdsec-firewall-bouncer.yaml
bouncers/crowdsec-nginx-bouncer.confIf you regenerate a bouncer API you need to make sure the new API is in the bouncer/yml file accordingly.
when everything is fine,
If you see errors in the crowdsec log, it's probably either machine not validated, wrong API number in lapi = crowdsec firewall service not running (main reason why often)
cscli machine list IP address need to be assigned, ortherwise it means the machine is not being used or there is an API issue.
cscli lapi status (success)
cscli capi status (success)
cscli metrics (return data)install linux/ssh/nginx/docker scenarios using cscli
sudo systemctl enable/start/status crowdsec-firewall-bouncer.service crowdsec.serviceneed to be healthy active/loaded at all time
Metabase
to handle the metabase feature, I simply run it this way:
cscli dashboard setup -l 127.0.0.1 -p 12345 --password YourPassw0rd
keep provided credentials (you can change them when logged)and then access the frontend first doing on a new terminal
ssh -L 3000:127.0.0.1:12345 user@hostand in the browser : http://localhost:3000
This leaves metabase isolated from the internet and only accessible via SSH
No Metabase
finally if you don't want to mess with having an "outside" cloudron metabase running to have a dashboard locally, just don't use cscli dashboard, if you want to remove it you can just cscli dashboard stop or remove
and then use cscli enroll command to have the crowdsec provided "as a service" dashboard https://app.crowdsec.net
I have 1 machine inactive in this setup because it is the one active on the nginx bouncer that is currently in place but not being used.
-
@mastadamus this is logical in fact, this is because the firewall bouncer intercept the traffic at the iptable level and stop 99.% of scenarios before traffic even hit the nginx process.
So if you really want to test the nginx bouncer you need to disable the firewall bouncer
@rmdes I tried nginx bouncer BEFORE installing iptables bouncer and it crashed my nginx and wouldn't let me bring it up. I then reverted to previous image and installed iptables bouncer which works great.
-
After more testing this is my report and steps to have it running smooth:
- adding crowdsec repo
- apt install crowdsec
- apt install crowdsec-firewall-bouncer-iptables
it should wizard the first init properly and add API keys accordingly, sometimes it seems the 2 services starts before the correct state of the whole crowdsec is correctly setup, so the solution is to (stop both services) make sure the list of follow up to the manual setup are correctly setup, the cscli tool has list, status and other commands to check if everything is fine.
I had to tweak crowdsec-firewall-bouncer.yml to this
Docker
I used this approach to get all the containers (acquis.yml)
# Docker context source : docker container_name_regexp: - ^[a-zA-Z0-9_.-]*$ labels: type: log_type ---I used port 9999 in config.yml and accordingly into
local_api_credentials.yaml
bouncers/crowdsec-firewall-bouncer.yaml
bouncers/crowdsec-nginx-bouncer.confIf you regenerate a bouncer API you need to make sure the new API is in the bouncer/yml file accordingly.
when everything is fine,
If you see errors in the crowdsec log, it's probably either machine not validated, wrong API number in lapi = crowdsec firewall service not running (main reason why often)
cscli machine list IP address need to be assigned, ortherwise it means the machine is not being used or there is an API issue.
cscli lapi status (success)
cscli capi status (success)
cscli metrics (return data)install linux/ssh/nginx/docker scenarios using cscli
sudo systemctl enable/start/status crowdsec-firewall-bouncer.service crowdsec.serviceneed to be healthy active/loaded at all time
Metabase
to handle the metabase feature, I simply run it this way:
cscli dashboard setup -l 127.0.0.1 -p 12345 --password YourPassw0rd
keep provided credentials (you can change them when logged)and then access the frontend first doing on a new terminal
ssh -L 3000:127.0.0.1:12345 user@hostand in the browser : http://localhost:3000
This leaves metabase isolated from the internet and only accessible via SSH
No Metabase
finally if you don't want to mess with having an "outside" cloudron metabase running to have a dashboard locally, just don't use cscli dashboard, if you want to remove it you can just cscli dashboard stop or remove
and then use cscli enroll command to have the crowdsec provided "as a service" dashboard https://app.crowdsec.net
I have 1 machine inactive in this setup because it is the one active on the nginx bouncer that is currently in place but not being used.
the main consequences I have seen is a drastic drop in all the wordpress (previously using wordfence or else) attacks/prob/vuln scan etc..after a week without these plugins doing anything (the firewall intercept these scenarios before they reach nginx) my wordfence logs are now bots/humans/myself, no more attacks.
it's just brutal since Armada host quite a few domains running wordpress, so basically the surface to attack drop's considerably with crowdsec agent monitoring the door, the Wordpress containers don't even get the bots usually hitting the domains, so I also have seen a drop in CPU/Mem usage overall everywhere, it's now running since 2 weeks without any problem.
I'm sure now that the first issue I had was because of the manual installation at that point in time, for some reason it pulled another version of nginx, but even if you reproduce these steps now, the installation just work fines and if you mess with the agent API, it's really easy to replace the API accordingly, have all the pieces of the YML files correctly configured, make sure systemd service are enabled and started and running and voila !
If you want to go the safe way, just use the Firewall (in fact if you use it, the nginx bouncer is irrelevant) in my experience setting up metabase the way I did is safe and does not impact cloudron in any way that I could see.
-
the main consequences I have seen is a drastic drop in all the wordpress (previously using wordfence or else) attacks/prob/vuln scan etc..after a week without these plugins doing anything (the firewall intercept these scenarios before they reach nginx) my wordfence logs are now bots/humans/myself, no more attacks.
it's just brutal since Armada host quite a few domains running wordpress, so basically the surface to attack drop's considerably with crowdsec agent monitoring the door, the Wordpress containers don't even get the bots usually hitting the domains, so I also have seen a drop in CPU/Mem usage overall everywhere, it's now running since 2 weeks without any problem.
I'm sure now that the first issue I had was because of the manual installation at that point in time, for some reason it pulled another version of nginx, but even if you reproduce these steps now, the installation just work fines and if you mess with the agent API, it's really easy to replace the API accordingly, have all the pieces of the YML files correctly configured, make sure systemd service are enabled and started and running and voila !
If you want to go the safe way, just use the Firewall (in fact if you use it, the nginx bouncer is irrelevant) in my experience setting up metabase the way I did is safe and does not impact cloudron in any way that I could see.
-
@rmdes that all sounds great. I wonder if Crowdsec could be baked into Cloudron itself?
@jdaviescoates I know the crowdsec people are working "upstream" with all kinds of actors to get this done for many hosting/implementations out there, in my experience, since both Cloudron and Crowdsec use iptables this integration is already the case, what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already :
- has a Docker type
- IPtables
- Can plug on any SystemD service to parse logs
what could probably be optimized is how crowdsec "watches" all the docker containers and perhaps make sure the implementation does not disrupt cloudron operations in any way, make it bullet proof install tested.
-
@jdaviescoates I know the crowdsec people are working "upstream" with all kinds of actors to get this done for many hosting/implementations out there, in my experience, since both Cloudron and Crowdsec use iptables this integration is already the case, what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already :
- has a Docker type
- IPtables
- Can plug on any SystemD service to parse logs
what could probably be optimized is how crowdsec "watches" all the docker containers and perhaps make sure the implementation does not disrupt cloudron operations in any way, make it bullet proof install tested.
@rmdes I am head of community at CrowdSec and we would be happy to work with anyone from the Cloudron community on this. For this purpose it would be a great idea to join our Discord community and talk about it there
-
@jdaviescoates I know the crowdsec people are working "upstream" with all kinds of actors to get this done for many hosting/implementations out there, in my experience, since both Cloudron and Crowdsec use iptables this integration is already the case, what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already :
- has a Docker type
- IPtables
- Can plug on any SystemD service to parse logs
what could probably be optimized is how crowdsec "watches" all the docker containers and perhaps make sure the implementation does not disrupt cloudron operations in any way, make it bullet proof install tested.
@rmdes said in Crowdsec Install guide for cloudron purposes:
what should be perhaps done is how the cloudron admin activate/onboard from the dashboard or something similar but I'm not sure about the added value since technically, it is integrated in many ways already
That would have massive added value to me.
I wouldn't trust myself to not mess something up doing it manually!
And anything that makes life easier and simpler is welcome. So just having some option in the Cloudron admin that could enable Crowdsec would be ideal.
-
@rmdes I am head of community at CrowdSec and we would be happy to work with anyone from the Cloudron community on this. For this purpose it would be a great idea to join our Discord community and talk about it there
@klausagnoletti I'm already there
but not a Cloudron dev or anything, but yeah it would be amazing the have the existing metabase cloudron app in "flavors" one could be the default app we have now, another one could bootstrap crowdsec db and be a one click install for the underlying crowdsec installed in the host but I have no idea if it's possible. -
N necrevistonnezr referenced this topic on
-
Warning
Please install at your own risk. Cloudron does not support installing extra applications and packages on the server.
Install crowdsec, IPtables bouncer, and log4j detection collection on cloudron and reconfigure cloudron nginx conf for default logging.
Install crowdsec
sudo curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash sudo apt install crowdsec(during the installation process Crowdsec install should install the appropriate "collections" which consists of parsers, and rules for the log sources on your cloudron.
Edit nginx.conf file to put in default logging.
- SSH into your cloudron
- using text editor of your choice open the nginx.conf file found at /etc/nginx/nginx.conf
- We want to changed the logging section to look like it does in this example nginx.conf file.
user www-data; # detect based on available CPU cores worker_processes auto; # this is 4096 by default. See /proc/<PID>/limits and /etc/security/limits.conf # usually twice the worker_connections (one for uptsream, one for downstream) # see also LimitNOFILE=16384 in systemd drop-in worker_rlimit_nofile 8192; pid /run/nginx.pid; events { # a single worker has these many simultaneous connections max worker_connections 4096; } http { include mime.types; default_type application/octet-stream; # the collectd config depends on this log format # required for long host names server_names_hash_bucket_size 128; access_log /var/log/nginx/access.log combined; sendfile on; # timeout for client to finish sending headers client_header_timeout 30s; # timeout for reading client request body (successive read timeout and not w hole body!) client_body_timeout 60s; # keep-alive connections timeout in 65s. this is because many browsers timeo ut in 60 seconds keepalive_timeout 65s; # zones for rate limiting limit_req_zone $binary_remote_addr zone=admin_login:10m rate=10r/s; # 10 req uest a second include applications/*.conf;Install bouncer
sudo apt install crowdsec-firewall-bouncer-iptablesInstall Log4j Collection
sudo cscli hub update sudo cscli scenarios install crowdsecurity/apache_log4j2_cve-2021-44228 sudo systemctl reload crowdsecInstallation of crowdsec metabase docker dashboard
- to install the docker container on port 8181(may change this as desired. (Note this should be for internal network access only.) Do not open this up to the internet. It may be better to try to integrate this with the metabase app that comes with cloudron available in the appstore. I haven't dug into that yet though. )
sudo cscli dashboard setup -l 0.0.0.0 -p 8181 --password < insert password>- To make persistent.
- Identify the crowdsec/metabase container ID number
docker ps- persistant command
sudo docker update --restart=unless-stopped <container ID number >Access metabase
- you can access metabase by navigating to "http://yourIP:8181
- your credentials will be crowdsec@crowdsec.net and whatever you set the password.
References:
https://docs.crowdsec.net/docs/getting_started/install_crowdsec
I am now checking out CrowdSec as well.
As of writing this the installation instruction from @Mastadamus where almost correct.
Edit nginx.conf file to put in default logging
Not needed, also the posted config is bad since it's missing the final closing curly bracket
}
Everything else worked as described.
Now to the bouncer.
@rmdes said in Crowdsec Install guide for cloudron purposes:
So if you really want to test the nginx bouncer you need to disable the firewall bouncer
Ahhh ok, nah I will stick with the iptable bouncer.
-
After more testing this is my report and steps to have it running smooth:
- adding crowdsec repo
- apt install crowdsec
- apt install crowdsec-firewall-bouncer-iptables
it should wizard the first init properly and add API keys accordingly, sometimes it seems the 2 services starts before the correct state of the whole crowdsec is correctly setup, so the solution is to (stop both services) make sure the list of follow up to the manual setup are correctly setup, the cscli tool has list, status and other commands to check if everything is fine.
I had to tweak crowdsec-firewall-bouncer.yml to this
Docker
I used this approach to get all the containers (acquis.yml)
# Docker context source : docker container_name_regexp: - ^[a-zA-Z0-9_.-]*$ labels: type: log_type ---I used port 9999 in config.yml and accordingly into
local_api_credentials.yaml
bouncers/crowdsec-firewall-bouncer.yaml
bouncers/crowdsec-nginx-bouncer.confIf you regenerate a bouncer API you need to make sure the new API is in the bouncer/yml file accordingly.
when everything is fine,
If you see errors in the crowdsec log, it's probably either machine not validated, wrong API number in lapi = crowdsec firewall service not running (main reason why often)
cscli machine list IP address need to be assigned, ortherwise it means the machine is not being used or there is an API issue.
cscli lapi status (success)
cscli capi status (success)
cscli metrics (return data)install linux/ssh/nginx/docker scenarios using cscli
sudo systemctl enable/start/status crowdsec-firewall-bouncer.service crowdsec.serviceneed to be healthy active/loaded at all time
Metabase
to handle the metabase feature, I simply run it this way:
cscli dashboard setup -l 127.0.0.1 -p 12345 --password YourPassw0rd
keep provided credentials (you can change them when logged)and then access the frontend first doing on a new terminal
ssh -L 3000:127.0.0.1:12345 user@hostand in the browser : http://localhost:3000
This leaves metabase isolated from the internet and only accessible via SSH
No Metabase
finally if you don't want to mess with having an "outside" cloudron metabase running to have a dashboard locally, just don't use cscli dashboard, if you want to remove it you can just cscli dashboard stop or remove
and then use cscli enroll command to have the crowdsec provided "as a service" dashboard https://app.crowdsec.net
I have 1 machine inactive in this setup because it is the one active on the nginx bouncer that is currently in place but not being used.
@rmdes said in Crowdsec Install guide for cloudron purposes:
I had to tweak crowdsec-firewall-bouncer.yml to this
Where? What?
https://docs.crowdsec.net/docs/bouncers/firewall/#configuration
/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yamlOk.
@rmdes said in Crowdsec Install guide for cloudron purposes:
I used this approach to get all the containers (acquis.yml)
Where? What?
https://docs.crowdsec.net/docs/v1.0/acquisitions/format/#acquisition-directives
/etc/crowdsec/acquis.yamlOk.
@rmdes said in Crowdsec Install guide for cloudron purposes:
I used port 9999 in config.yml
Where? What?
https://docs.crowdsec.net/docs/configuration/crowdsec_configuration#configuration-directives
/etc/crowdsec/config.yamlIn yaml querry / json qurry format:
yq '.api.server.listen_uri' < /etc/crowdsec/config.yaml
127.0.0.1:8080this changed to127.0.0.1:9999I assume.
Changed to:9999@rmdes said in Crowdsec Install guide for cloudron purposes:
and accordingly into
local_api_credentials.yamlWhere? What?
https://docs.crowdsec.net/docs/next/local_api/intro/
/etc/crowdsec/local_api_credentials.yamlIn yaml querry langauge:
yq '.url' < /etc/crowdsec/local_api_credentials.yaml
http://127.0.0.1:8080changed to:9999@rmdes said in Crowdsec Install guide for cloudron purposes:
bouncers/crowdsec-firewall-bouncer.yaml
Where? What?
https://docs.crowdsec.net/docs/bouncers/firewall/
/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yamlyaml querry:
yq '.api_url' < /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
http://127.0.0.1:8080changed to:9999@rmdes said in Crowdsec Install guide for cloudron purposes:
install linux/ssh/nginx/docker scenarios using cscli
What? How?
https://docs.crowdsec.net/docs/v1.1/cscli/cscli_scenarios_install/
cscli scenarios install
Which scenarios can I install?
https://hub.crowdsec.net/browse/Following your advice:
cscli scenarios install linux cscli scenarios install sshd cscli scenarios install nginxI did not find and scenario for docker? @rmdes and now when I try
cscli scenarios install linuxI get an error.
Well it seems this are not scenarios but collections.So it's https://docs.crowdsec.net/docs/v1.2/cscli/cscli_collections_install/
cscli collections install crowdsecurity/linuxWhich is already installed by default, which also included
sshdby default.I could not find docker at all.
Maybe you can help out here @rmdes -
@rmdes said in Crowdsec Install guide for cloudron purposes:
I had to tweak crowdsec-firewall-bouncer.yml to this
Where? What?
https://docs.crowdsec.net/docs/bouncers/firewall/#configuration
/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yamlOk.
@rmdes said in Crowdsec Install guide for cloudron purposes:
I used this approach to get all the containers (acquis.yml)
Where? What?
https://docs.crowdsec.net/docs/v1.0/acquisitions/format/#acquisition-directives
/etc/crowdsec/acquis.yamlOk.
@rmdes said in Crowdsec Install guide for cloudron purposes:
I used port 9999 in config.yml
Where? What?
https://docs.crowdsec.net/docs/configuration/crowdsec_configuration#configuration-directives
/etc/crowdsec/config.yamlIn yaml querry / json qurry format:
yq '.api.server.listen_uri' < /etc/crowdsec/config.yaml
127.0.0.1:8080this changed to127.0.0.1:9999I assume.
Changed to:9999@rmdes said in Crowdsec Install guide for cloudron purposes:
and accordingly into
local_api_credentials.yamlWhere? What?
https://docs.crowdsec.net/docs/next/local_api/intro/
/etc/crowdsec/local_api_credentials.yamlIn yaml querry langauge:
yq '.url' < /etc/crowdsec/local_api_credentials.yaml
http://127.0.0.1:8080changed to:9999@rmdes said in Crowdsec Install guide for cloudron purposes:
bouncers/crowdsec-firewall-bouncer.yaml
Where? What?
https://docs.crowdsec.net/docs/bouncers/firewall/
/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yamlyaml querry:
yq '.api_url' < /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
http://127.0.0.1:8080changed to:9999@rmdes said in Crowdsec Install guide for cloudron purposes:
install linux/ssh/nginx/docker scenarios using cscli
What? How?
https://docs.crowdsec.net/docs/v1.1/cscli/cscli_scenarios_install/
cscli scenarios install
Which scenarios can I install?
https://hub.crowdsec.net/browse/Following your advice:
cscli scenarios install linux cscli scenarios install sshd cscli scenarios install nginxI did not find and scenario for docker? @rmdes and now when I try
cscli scenarios install linuxI get an error.
Well it seems this are not scenarios but collections.So it's https://docs.crowdsec.net/docs/v1.2/cscli/cscli_collections_install/
cscli collections install crowdsecurity/linuxWhich is already installed by default, which also included
sshdby default.I could not find docker at all.
Maybe you can help out here @rmdesNow I am running crowdsec on my cloudron.
I will report back for more findings.Like my work? Consider donating a drink. Cheers!
-
Now I am running crowdsec on my cloudron.
I will report back for more findings.@BrutalBirdie Thanks! It would be great if your, @rmdes and @Mastadamus findings could end up in a Cloudron doc in the end...!
-
Now I am running crowdsec on my cloudron.
I will report back for more findings.@BrutalBirdie Sorry have been busy and thanks for documenting my steps
The docker part is done on the /etc/crowdsec/acquis.yml
I'm not entirely convinced my regex call works appropriately
or even that in the cloudron context this is entirely needed (I have one cloudron with & without not seeing much difference)# Docker context source : docker container_name_regexp: - ^[a-zA-Z0-9_.-]*$ labels: type: log_type --- -
Revisiting this topic - is Crowdesc running reliably for those who got it installed?
-
-
L lukas referenced this topic on
-
-
not anymore, there has been a lot of changes on crowdsec side and not only the crowdsec-firewall-bouncer needs a very specific nginx version but even the wordpress bouncer could not be properly installed last time I checked.
I still have one instance running behind a Cloudflare, a server running cloudron + crowdsec but without the nginx bouncer, just the cloudflare bouncer, but even in this configuration I could not get the wordpress bouncer to work.
-
