Re: Routing with multiple IPs

From: P Gentry (rdgentry1_at_cablelynx.com)
Date: 03/26/04 

Date: 26 Mar 2004 11:55:28 -0800

lfernando@pini.com.br (Luiz Fernando de F. F.) wrote in message news:<6f25de76.0403241254.28f6218@posting.google.com>...
> Hello folks,
>
> I've got a routing problem that's more or less covered on the URL
> below:
>
> http://lartc.org/howto/lartc.rpdb.multiple-links.html
>
> But that's not exactly my figure. Instead of having 2 links to
> different ISPs, I've got 2 links (as in IPs) with the same ISP, on ...

Do you have two _physical_ links or just 2 IPs given out by your ISP?
I assumed just one link, but your description below suggests 2 links
to your ISP. MASQing will not work wtih a 2 link setup to your ISP.

> ... different NICs, as below (fictional numbers, of course):
>
> eth0 - 192.168.0.1/24 (LAN interface)
> eth1 - 10.0.0.2/27 (ISP interface)
> eth2 - 10.0.0.3/27 (ISP interface)
>
> The ISP router is plugged in my hub, and all the NICs on my box are
> plugged on this hub too. ...

This makes no sense or requires the extra giant size aspirin bottle!
Are we talking of one machine with all nics plugged into hub? Must be
two machines -- one is the ISP router, the other has 3 nics (one nic
to router, one to DMZ, one plugged into hub). I can't untangle this
verbal ambiguity.

> .... What I want is to use the eth1 interface for
> masquerading my LAN (eth0). The eth2 interface will be for the
> webserver (that will be on the LAN, I'll later do a portforward/DNAT
> to it), and maybe other stuff on the future (ftp, vpn, etc.).

Since I'm already confused, the rest only makes it worse. With
routing (and especially arp) setup problems you _must_ be explicit to
the slightest detail. An accurate, detailed description/drawing of
the physical layout is required. Verbatim copies of command input
together with verbatim copies of output (and which box it was entered
on) to get info on routing tables and interfaces (route -n and
ifconfig) from _each_ box. Same for ping and traceroute. Verbal
descriptions are like running into a spider web in the dark! I feel
like I've run into 3 or 4 already -- I give up.

> The masquerading goes ok, but when I ping (from outside) the 10.0.0.3
> ip, on the box the ping comes through the eth1 interface, when it
> should come from the eth2 interface, and the reply also goes out
> through the eth1 interface. I tried using the routing explanations
> given on the URL above (with some adaptations for my case), thinking
> it was some routing problem, and while the reply "tries" to go through
> the eth2 interface, the ping request still comes from the eth1
> interface.
>
> My best guess is that it is some ARP problem... when I ask for the arp
> table, the reply is:
>
> Address HWtype HWaddress Flags Mask Iface
> 192.168.0.10 ether 00:50:DA:19:A7:EF C eth0
> 10.0.0.1 ether 00:E0:1E:B8:F9:34 C eth1
>
> 192.168.0.10 is the webserver in my lan, to which I'm
> portforwarding... 10.0.0.1 is my ISP's router. Those entries are
> correct, but my ISP's router entry is especified only for the eth1
> interface. Isn't there supposed to be another entry for the eth2
> interface? I tried to manually add it, with "arp -Ds 10.0.0.1 eth2",
> but I was remotely configuring the machine and I lost the connection
> after that, had to call the people on the place to restart the server.
>
> People on IRC suggested that I use both my ISP's IPs (10.0.0.2 and
> 10.0.0.3) on only one NIC, but I'm reluctant to do it because then I
> can only filter by IP on the firewall, and I've seen on the iptables
> howto this is also discouraged, because it's less secure (because of
> the inherent reduced control over the packet filtering). Anyone
> disagrees? Or have any clues to my problem?

I'm so confused as to your setup/layout that I can't comment.

> Thanks!
>
> Luiz Fernando

Willing to help, but I just spent 40 minutes trying to understand your
setup -- both physical and logical (IP) -- and I am clueless. More
now than when I started.

What kind of ISP feed do you have? dsl? cable modem? frame relay?
ISDN?

What does the ISP feed connect to?

Your "ISP router" belongs to you? to your ISP? Has one interface to
ISP feed and one other interface to proper, upstream hub port?

You have a second machine with 3 nics (2 with ISP IPs and 1 with LAN
IP) and all nics are plugged into downstream hub ports? I must be
wrong -- this makes no sense.

Start again ...

The "ISP router" is yours and has 3 nics (2 with ISP IPs and 1 with
LAN IP). The LAN interface (eth0) is plugged into the upstream port
of the hub and the rest of your LAN boxes are plugged into the
downstream ports. Yes? For sure, one ISP interface is connected to
ISP feed -- the other ISP interface has a separate physical feed? (If
so, MASQing will not work.)

You need something along these lines, I think -- could be wrong,
certainly I'm still confused.

ISP feed -> Your border router with 3 nics (2 with ISP IPs and 1 with
LAN IP) --- ...
... -> LAN interface to upstream hub port -> downstream hub ports feed
rest of LAN

Your border router's setup:
the eth1 connects to ISP feed
the eth2 connects to your DMZ (and routes through eth1 to reach ISP)
the eth0 connects to hub --> connects to LAN

Depending on what you are looking for, you can place a second, MASQing
router between eth0 and hub -- recommended -- or try to set up MASQing
between eth0 and eth1 -- you cannot MASQ to two outbound interfaces --
and you don't want SYN packets from eth1 or eth2 to eth0, so you will
need a firewall here as well to filter packets.

I have no idea where your reluctance to filter packets at the ISP-LAN
router comes from -- it's wrong. You want to filter packets as soon
as possible or sooner! You can filter packets on any machine that has
packet filtering enabled -- both at the border and inside your LAN.
This is GOOD! Filtering _only_ after packets have come in -- this is
BAD!

Though I remain befuddled, I think your design/layout needs very
serious rethinking.

hth,
prg
email above disabled

Relevant Pages