Entering too many IP addresses to be blocked at network level by Cloudron results in error: "setBlocklist (stderr): ipset v7.15: Hash is full, cannot add more elements"
-
I have a client site that's getting a ton of spammy user registrations which I'm finding difficult to clean up but more importantly preventing it in the first place.
I know there are a few spam-related plugins that could help, but I figured since it may affect other areas too like email spam and other client sites seeing as these IPs are hijacked / maliciously used, I would try to use the StopForumSpam IP blocklist to block traffic (I compared the IPs that are hitting the registration endpoint and they exist on the list.
However when I try to paste these IPs and save it, I get an error which shows as this in the logs (and the IPs are not added):
Nov 29 00:07:06box:shell setBlocklist (stderr): ipset v7.15: Hash is full, cannot add more elements
-
For the record, the three main IPs I'm seeing frequently are the following and all present as blocklisted at CleanTalk and StopForumSpam resources:
- 5.42.64.62
- 37.139.53.17
- 46.161.15.14
I mention this in case others are seeing this too. May be a good reminder to update those network block lists.
-
@d19dotca said in Entering too many IP addresses to be blocked at network level by Cloudron results in error: "setBlocklist (stderr): ipset v7.15: Hash is full, cannot add more elements":
Blocked IPs & Ranges 122095 IP(s) blocked
I started looking into this and something seems off. https://raw.githubusercontent.com/X4BNet/lists_stopforumspam/main/ipv4.txt itself only has only 4267 addresses. I guess you are adding lot more addresses? But 120k addresses seems like a lot.
-
@girish said in Entering too many IP addresses to be blocked at network level by Cloudron results in error: "setBlocklist (stderr): ipset v7.15: Hash is full, cannot add more elements":
I started looking into this and something seems off. https://raw.githubusercontent.com/X4BNet/lists_stopforumspam/main/ipv4.txt itself only has only 4267 addresses. I guess you are adding lot more addresses? But 120k addresses seems like a lot.
Oh that’s a great catch! I’m not sure how that happened. Let me test again with clearing out the setting entirely and saving blank, then pasting them in again and see what it says.
-
@d19dotca One thing I found is that when adding the stopforumspam list and the firehol list together, my system gets hosed! I will debug this further tomorrow. As in, some iptables/ipset issue. Maybe this needs a lot more memory at kernel level.
-
Just throwing this out there for what it is worth... I started experimenting, but got side tracked a while back. But I was looking at using CrowdSec to block traffic. There is at least a few Firhol lists and some other items.
It would be interesting to create one for forums and consolidate data from some of the other sources to create a blocklist. I did this to offset the "blocking" to an outside source, so that my server didn't quite take the hit.
-
@JLX89 There‘s this guide: https://forum.cloudron.io/topic/6224/crowdsec-install-guide-for-cloudron-purposes but I‘m not sure it‘s still up-to-date - and I‘d very much prefere to have such functionality built-in
-
@necrevistonnezr Yeah, I've seen that before. I'm looking a few things over, hopefully today -- see what I can come up with.
-
OK, i have debugged this now. The issue was that Firehol lists includes both 127.0.x.x and the 172.16.x.x networks which ends up blocking database and docker access . Commenting out those two works fine.
So, @d19dotca the issue was really just that you have somehow pasted 120k addresses. The max size of ipset is 65536. I will put a validation check in place but otherwise there is no bug.
-
-
@girish said in Entering too many IP addresses to be blocked at network level by Cloudron results in error: "setBlocklist (stderr): ipset v7.15: Hash is full, cannot add more elements":
The max size of ipset is 65536. I will put a validation check in place but otherwise there is no bug.
This may mean that using popular blocklists like this one won't work then (there is over 120k IPs, so I don't think it was a mistake earlier btw as you can see the number of unique IPs in the header comments on this list): https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/stopforumspam_90d.ipset
Perhaps this can be looked at for expansion in the future?
I was able to use their 30 day one (which still included the few IPs that seem persistent for my WordPress sites as trying to spam them) thankfully though as it's less than the limit in Cloudron.
-
-