New Firewall Feature issues

[savity]

Hi,
was happy about the new Firewall Feature so i started Blocking by Countrys.
Blocked China and Russia IPs.
So then at a Point i get the message Response timeout.
Now i am not abel to Delete any IP or add new IP. I always get the Response timeout.

So i checked the box.log it tells me the IP's are duplicated
Box POST /api/v1/network/blocklist 500 Internal Server Error Response timeout 20001.781 ms - 72
2020-09-12T07:43:00.057Z box:shell setBlocklist (stdout): ipset v6.34: Element cannot be added to the set: it's already added

2020-09-12T07:43:00.060Z box:shell setBlocklist code: 1, signal: null
ERROR HttpError: Error setting blocklist: setBlocklist exited with code 1 signal null
at Function.BoxError.toHttpError (/home/yellowtent/box/src/boxerror.js:97:16)
at /home/yellowtent/box/src/routes/network.js:29:41
at /home/yellowtent/box/src/network.js:45:27
at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:69:9)
at ChildProcess.emit (events.js:198:13)
at ChildProcess.EventEmitter.emit (domain.js:448:20)
at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12) [ /home/yellowtent/box/node_modules/express/lib/application.js:630:43 ]

So i deleted the List and pasted everything again now i am getting

Box POST /api/v1/network/blocklist 500 Internal Server Error Response timeout 20001.588 ms - 72
ERROR [object Object] [ /home/yellowtent/box/node_modules/express/lib/application.js:630:43 ]

I will check the list on duplicates but, is there any Limits on IP Addresses or how much you can configure for the first time?
Best Regards

[girish]

@savity Thanks for the report. Looks like we have a couple of bugs here:

• Cloudron doesn't handle duplicates nicely.
• For large lists, maybe it's taking a lot of time to add them. So, the request times out. It could also be that the request is large enough that it hits some API limits.

Can you tell me roughly how big the request is? A rough way to measure this is to save the IPs in a file and check the size of the file.

[girish]

@savity said in New Firewall Feature issues:

is there any Limits on IP Addresses or how much you can configure for the first time

There's no limits from Cloudron side intentionally, but there are probably API limits we are hitting. So we have to adjust our code accordingly to allow large request sizes.

[nebulon]

The dedupe is now fixed.

[savity]

First i Importet

https://www.ipdeny.com/ipblocks/

RUSSIAN FEDERATION (RU) [download ru.zone] Size: 170 KB (10497 IP blocks)

And after Importing

CHINA (CN) [download cn.zone] Size: 136 KB (8444 IP blocks)

The issues startet

[girish]

I was to go upto 18k in mine. I will keep adding.

[girish]

I could finally reproduce this after adding 30k blocks.

Box POST /api/v1/network/blocklist 500 Internal Server Error Response timeout 20001.547 ms - 72
ERROR [object Object] [ /home/yellowtent/box/node_modules/express/lib/application.js:630:43 ]

[savity]

Looks like it depends on the server ressources maybe.

[girish]

@savity I have fixed this for the next release. You can add this one line - https://git.cloudron.io/cloudron/box/-/commit/467fa590230853e426710e73e467c1f2596f9f86 in /home/yellowtent/box/src/routes/network.js and systemctl restart box after that.

[girish]

For, 5.6.1, I also migrated the block list from a json format to a simple text file format. This allows us to have comments, blank lines etc which is useful when copy/pasting all these block lists.

[savity]

thank youuuuu