Script to check nginx access logs for 403, 404 and check IP's with greynoise API then send to cloudflare.

[robi]

Can you modify this for directly adding to the Cloudron IP block list?

[Mastadamus]

@robi let me look into it. Need to see how I would interact with that list.

[fbartels]

@Mastadamus slightly off topic, but how does greynoise compare to crowded?

[Mastadamus]

@fbartels greynoise just runs a big honeypot and sensor landscape that captures and catalogues internet scanning. whereas crowdsec is a community sourced reporting of both scanning and malicious actions. Greynoise I believe has a much larger dataset. Its primary purpose is to see if an IP is targeting just you or multiple entities.

[fbartels]

@Mastadamus ah, very interesting.

[micmc]

@Mastadamus Sounds enough amazing to me, and with @robi 's suggestion that would even be greater, now wouldn't a script in the like be possible for Apache as well?

I mean I run wp plugin on my wp sites that detects all 404 like I've never seen before and many of them are obviously scans trying to find ways to attein some presumably installed backend scripts or even the .env file in directly in documentroot. When I discovered that is where I thought that something in the like of what you described here above, would be very useful.

[Mastadamus]

@micmc Yes you could easily take this script and use it for apache. The only think you need to do is look at your apache logs and see what position the source IP is in. the parts in the script where it does awk '{print $1}' etc. are telling it to grab the first position in the nginx log which happens to be the source IP for my particular logging configuration. so really the only think you would need to tailor/alter is the awk statements.

[micmc]

@Mastadamus
Sounds great, will take a closer look then thanks a lot mate.

[Mastadamus]

@girish If I add IP's to /home/yellowtent/platformdata/firewall/blocklist.txt will the automatically be blocked or will I need to restart the box service?

[girish]

@Mastadamus systemctl restart cloudron-firewall will read that file and apply the rules.

(As a warning, this file gets re-written if you go to Network -> Block addresses. So, you might lose your changes)

[Mastadamus]

@girish Gotcha but as long as I don't do that, I should be gtg. Im just thinking of a script that does 3 things.

1. Grabs all the IP's from emerging threats block list
2. Grabs all the 403/404's from access logs sends them to greynoise to check if they are known "noise" and then
3. Add both of these IP groups to that file and restart the service.