How to proctect instances from Bot, Crawlers, Requests, & Co?

[joseph]

Is there a service out there that provides AI block lists? crowdsec only has a platinum list at 3500/month ?

[p44]

@jdaviescoates @joseph Thanks, I wouldn't want to rely on outside services.

[jdaviescoates]

@p44 TBH wrt WordPress i'd expect Wordfence would likely do a much better job that you'd ever be able to do manually. They have very long blocklists and know about many more bad IPs than you do. I just use the free version.

[p44]

@jdaviescoates Thanks, it seems that they don’t block AI bots...

[robi]

Can you extract the blocklists from Wordfence to have it populate the Cloudron deny list?

[p44]

@robi Thanks for advice.

I don’t know where and if Wordfence has a public list, but I think that blocklists has a lot of data that can be huge to handle from CPU.

[robi]

@p44 those are solved problems with lookup tables.

[jdaviescoates]

@p44 I think Wordfence adds stuff to .htaccess

[p44]

@robi Thanks a lot.

I found and applied specific rules in Wordpress .htaccess:

• 8G Firewall
• Ultimate Block List to Stop AI Bots

I think is a good start.

Both filters it seems working fine. Of course, it would better to manage and deploy centrally.

Thanks again for your advices Robi.

[robi]

@p44 You are very welcome.

Now making a tool to parse those IPs for the Cloudron block list is something an eager LLM agent could do.

[p44]

@robi Amazing!!!

Something like a “bridge” that every day take from the source and update Cloudron block list...