Firewall / Spamassassin: Automatic list update

[necrevistonnezr]

I guess since the blocklist is in /home/yellowtent/platformdata/firewall/blocklist.txt, one could build something with a script & cron?

[necrevistonnezr]

@girish is the way I described feasible? Is that txt file the actual list the firewall accesses to check blocked IPs or is this txt file e.g. used to feed into a database?

[girish]

@necrevistonnezr Updating the txt file is not enough. The txt file is actually just a "cache" , the real value is stored in the database.

[necrevistonnezr]

@girish Ok, so simple scripting is out of the question.

[girish]

@necrevistonnezr you can still use the api though

[necrevistonnezr]

Well, the "setBlockList" operation allows to add a range of IPs but not a list of IPs in a file or am I wrong?

curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $CLOUDRON_TOKEN" "https://$CLOUDRON_DOMAIN/api/v1/network/blocklist" --data '{"blocklist":"# Spammy network\n10.244.0.0/16"}'

as per: https://docs.cloudron.io/api.html#tag/Network/operation/setBlockList

[girish]

@necrevistonnezr it's a "Newline separated list of IP entries" . So, it can be # Spammy network\n10.244.0.0/16\n1.2.3.4\n3.4.5.6\n172.4.0.0/16

[necrevistonnezr]

@girish I guess there's no mechanism to avoid duplicate entries when using the "setBlockList" operation, correct?

In general, I guess something like this should work:

#!/bin/bash
curl https://www.ipdeny.com/ipblocks/data/countries/kz.zone --output iplist.txt
while read -r line; do
    curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $CLOUDRON_TOKEN" "https://$CLOUDRON_DOMAIN/api/v1/network/blocklist" --data $line"
done < iplist.txt
rm iplist.txt

I don't have know yet how to avoid duplicates in the database..

[imc67]

It should be default functionality to have country block/allow in the Cloudron GUI just like all Synology NAS’s have. It’s 2023 and too dangerous to have everything accessible for everyone. That’s why many Cloudron users (read the forum) are using Cloudflare for this kind of functionality (like I have to do).

[girish]

@imc67 IMO, the correct place to implement this is in the network firewall. Most Cloud providers already have a firewall feature and they can then implement this firewall rule at the edge of the network instead of the server itself.

I have a Synology router (not NAS) at home. I just use their blocklists. For home setups, the router is the correct place for this. Otherwise, you allow all traffic to come into your home and then it gets rejected by the server wasting cpu and network traffic.

That said, I understand why this feature is being requested here instead - no cloud network firewall has this feature. And most likely cloud providers don't listen to our suggestions

[robi]

@girish IME it's a custom support request for the network operator to put those filters on for your IP(s).

[d19dotca]

Is it possible to improve / add the IP block list feature to Cloudron in 8.0?

[imc67]

@necrevistonnezr said in Firewall / Spamassassin: Automatic list update:

I guess something like this should work

Did you managed it to get it worked like that?

[necrevistonnezr]

@imc67 No, as I don’t know how to avoid duplicates in the database - I don’t want to fill up a database with a hard entry limit with nonsense

[imc67]

@necrevistonnezr I gave it a try with some help by ChatGPT and it works flawless!! Except the API can't handle large list where the GUI is able to handle without an issue.

The script automatically downloads all the geo lists in an array, creates a copy/paste file for the GUI and then prepares the file in JSON style and connect/upload via API.

When I choose only a few countries is works perfect, however when choosing all the desired ones:

@girish @nebulon I get a line 83: /usr/bin/curl: Argument list too long

I can copy/paste the full list in the GUI, it takes some time but it uploads and settles all IP ranges (about 87k)

Does anyone know how to do this via the API?

[robi]

@imc67 Since it works from the browser, you should be able to inspect the API calls in the browser console.

[girish]

@imc67 said in Firewall / Spamassassin: Automatic list update:

@girish @nebulon I get a line 83: /usr/bin/curl: Argument list too long

this is related to curl . I don't know the answer but you can look for posts similar to https://stackoverflow.com/questions/54090784/curl-argument-list-too-long where you have to pass the args as a file instead of on the command line itself.

[imc67]

@girish said in Firewall / Spamassassin: Automatic list update:

https://stackoverflow.com/questions/54090784/curl-argument-list-too-long

I tried that in the beginning but got this error:

{
  "status": "Bad Request",
  "message": "blocklist must be a string"
}

[imc67]

@robi I did a check and somehow the browser is able to do a POST with the huge string ...

[imc67]

Again ChatGPT did it!!! It's really unbelievable how "patient" it is and after keep trying all it's variants (after feedback the errors) and asking if there is another way except curl it came with wget and after the second try it worked!