Re: loading huge number of rules in iptables (blocklist)

Andrew Sackville-West wrote:
On Wed, Mar 21, 2007 at 02:30:06PM -0400, H.S. wrote:


okay, I follow... and you want otherwise unfettered p2p operating, but
security from these particular sites. ugh. nasty problem.

Nasty problem, yes. But I can live without it since I don't do much p2p.
But the problem did present me a chance of reading IP ranges from a file
and blocking them. I have been playing around with this idea for
blocking some IP addresses that seem to bombard you with ssh attempts.
But this is probably overkill for that and is surely to give many false
positives (Using a differnt port that 22 seems to be easiest solution).


The result was the experiment to use the massive blocklist and to
automate the process in iptables firewall on a router -- needs iptables,
bash, curl and maybe pythong or perl. I am giving it a shot. As I said
before, this is the first attempt.

so, is there some other way to use this info besides a massive
iptables rule set? I'm in territory I don't understand, so feel free
to ignore me ;). What about a proxy? instead of a ruleset in the
firewall, run the whole thing through a proxy that is set up to read
the list of denies. then a simple update of the list can result in new
blocking without reloading a whole set of rules. I don't have a clue
as to the mechanics of this.


I am already playing with similar idea: but it needs to bootstrap the
iptabels rules once. After that, I can work out a script that downloads
the ipranges block list and diffs it with the previous one and deletes
or inserts the different rules. However, even starting up the rules once
is taking impractically long (I think it will take around 2~3 days to
load all the rules on my old machine).


Another possibility, from my reading it appears this is supposed to
work with a program called moblock. moblock seems to do some parsing
to eliminate duplicates and so forth. I've done a little grepping
through the list and can can tell that it could be done more
efficiently. First, there are duplicates as shown here:

andrew@basement:~$ zcat level1.gz | wc -l
151663
andrew@basement:~$ zcat level1.gz | cut -d: -f2 | uniq | wc -l
150695

that's 1000 rules gone right there. (the cut eliminates the name,
giving just the ip range.

Interesting.


Also, a little scripting could probably concatenate a lot of the
ranges. just a cursory look through shows that there are contiguous
ranges specified on different lines. I don't have time today to hack
at it, but I think you might be able to cut as much as 25% out of the
list that way.

25% reduction won't cut it, I am afraid. If we are talking about 80%
reduction in the eventual number of rules, then it is workable. Or a
different method is needed to make up these many rules.

I am not going to follow up on my current method. A better one is
definitely needed.

In any case, an interesting experience.
->HS


--
To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx
with a subject of "unsubscribe". Trouble? Contact listmaster@xxxxxxxxxxxxxxxx