Linux multihomed routing (2 ISP, 1 internal network) problem

Hi all !!

I'm having a little trouble with a multihomed setup I am running here.
I'll try to give as much information as possible.

I am using the latest Debian release with kernel 2.6.18-4-686.
I am doing the routing using iproute2. the rules are setup with
iptables.
The system has 3 NICs: eth0 has the cable ISP (called "TELENET"), eth1
has the ADSL ISP (called "SKYNET") (with external modem) and eth2 is
my internal network (called "INTERN"). The ADSL line has a fixed IP,
and is used to connect to one of our servers from remote locations
(the modem is setup to forward everything that hits it to my IP on
eth1)
The cable ISP is the preferred ISP (ie: all undefined outgoing traffic
goes thru the cable provider).
All clients on the lan have full access, no restrictions, to do
whatever they want online.
I am using ip route rules and iptables with --set-mark to tell certain
traffic which routing table to use.

When I finished configuring Saturday evening, everything was working
like a charm.
When I tested again today, suddenly it didn't work as good anymore ...
Something has changed overnight but I have no clue what it is.
After some testing, I figured out 2 things: my dns traffic suddenly
wants to go thru the DSL line and my clients can not directly connect
to the internet anymore. After setting a clients dns servers to the
DSL ISP's dns servers, they could resolve again, but they can still
not surf anymore. The Linux machine itself can still perfectly do
everything.

I hope somebody can help me out here, because I am at a loss; I've
once been able to set this up about 6 years ago, but that knowledge
has faded a bit (and isn't apparantly very useful anymore anyways). I
am also not a linux noob ;-)

Below you can find all technical stuff.

Thanks to all of you in advance for your help !!

Cheers,
Andy


------------ MAIN ROUTING TABLE -----------
# ip route show table main
192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.254
192.168.254.0/24 dev eth1 proto kernel scope link src 192.168.254.2
81.82.0.0/19 dev eth0 proto kernel scope link src 81.82.x.x
default via 81.82.0.1 dev eth0

------------ EXTRA ROUTING TABLE-----------
# ip route show table 4
192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.254
192.168.254.0/24 dev eth1 proto kernel scope link src 192.168.254.2
81.82.0.0/19 dev eth0 proto kernel scope link src 81.82.x.x
default via 192.168.254.1 dev eth1

----------- ROUTING RULES -----------
# ip rule show
0: from all lookup 255
32764: from 192.168.254.1 lookup 4
32765: from all fwmark 0x4 lookup 4
32766: from all lookup main
32767: from all lookup default

---------- FIREWALL (rules) SCRIPT (partial) ----------

IPTABLES=/sbin/iptables
TELENET="eth0"
SKYNET="eth1"
INTERN="eth2"
INTNET="192.168.0.0/24"
$IPTABLES -F
$IPTABLES -F -t nat
$IPTABLES -F -t mangle
$IPTABLES -A INPUT -i lo -s 127.0.0.1/8 -d 0.0.0.0/0 -j ACCEPT
$IPTABLES -A OUTPUT -o lo -s 127.0.0.1/8 -d 0.0.0.0/0 -j ACCEPT
$IPTABLES -A INPUT -i $TELENET -s 0.0.0.0/0 -d $TELENETIP -m state --
state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A INPUT -i $SKYNET -s 0.0.0.0/0 -d $SKYNETIP -m state --
state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A OUTPUT -o $TELENET -s $TELENETIP -d 0.0.0.0/0 -j ACCEPT
$IPTABLES -A OUTPUT -o $SKYNET -s $SKYNETIP -d 0.0.0.0/0 -j ACCEPT
$IPTABLES -A INPUT -i $INTERN -s $INTNET -d 0.0.0.0/0 -j ACCEPT
$IPTABLES -A OUTPUT -o $INTERN -s $INTERNIP -d $INTNET -j ACCEPT
$IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
443 -j MARK --set-mark 0x4
$IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
444 -j MARK --set-mark 0x4
$IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
1723 -j MARK --set-mark 0x4
$IPTABLES -t mangle -A PREROUTING -s $SERVER1IP -p tcp -m tcp --sport
4125 -j MARK --set-mark 0x4
$IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 443 -
m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
$SERVER1IP:443
$IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 444 -
m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
$SERVER1IP:444
$IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 1723
-m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
$SERVER1IP:1723
$IPTABLES -t nat -A PREROUTING -d $SKYNETIP -p tcp -m tcp --dport 4125
-m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination
$SERVER1IP:4125
$IPTABLES -t nat -A POSTROUTING -o $TELENET -j SNAT --to-source
$TELENETIP
$IPTABLES -t nat -A POSTROUTING -o $SKYNET -j SNAT --to-source
$SKYNETIP
$IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
1024:65535 --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
$IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
1024:65535 --dport 444 -m state --state NEW,ESTABLISHED -j ACCEPT
$IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
1024:65535 --dport 1723 -m state --state NEW,ESTABLISHED -j ACCEPT
$IPTABLES -A INPUT -d $SKYNETIP -i $SKYNET -p tcp -m tcp --sport
1024:65535 --dport 4125 -m state --state NEW,ESTABLISHED -j ACCEPT
$IPTABLES -A FORWARD -d $INTNET -j ACCEPT
$IPTABLES -A FORWARD -s $INTNET -j ACCEPT
$IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 443 -
m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 444 -
m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 1723 -
m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -A FORWARD -i $SKYNET -o $INTERN -p tcp -m tcp --dport 4125 -
m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -P FORWARD DROP
$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT DROP

.

Relevant Pages