Re: iptables + multiple gateways not firewalling

On Sat, Dec 12, 2009 at 1:49 PM, Ian Coetzee <ubuntu@xxxxxxxxxxxxxxxxx> wrote:
On Sat, Dec 12, 2009 at 1:36 PM, Werner Schram <wrschram@xxxxxxxxx> wrote:
Hi Ian,

Ian Coetzee wrote:
Hi all

I have a small problem here (I may have just overlooking something here)

I have a ubuntu 9.10 server here. On it I do web proxing through one
provider, and mail through another.

I had it set up with one NIC and it worked (in my testing
environment). However I am unable get get it to work, after I inserted
another NIC to route the mail out through.

I *think* I narrowed it down to the fact that ip tables is not
firewalling eth1(which is the new NIC I inserted).

I can ping both my NIC's subnets that is connected to them.

Any help will be greatly appreciated.

My config is as such (public IPs is changed due to lurking eyes :) )

eth0 (internal) -> 192.168.1.3/24 gateway 192.168.1.15
eth1 (external) -> 111.111.111.52/29 gateway 111.111.111.49

iptables rules:
 cat /etc/iptables.up.rules
# Generated by iptables-save v1.4.4 on Sat Dec 12 11:02:07 2009
*mangle
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
#mark outgoing tcp/25 packets for routing
-A OUTPUT -p tcp -m tcp --sport 25 -j MARK --set-mark 0x10
COMMIT
# Completed on Sat Dec 12 11:02:07 2009
# Generated by iptables-save v1.4.4 on Sat Dec 12 11:02:07 2009
*nat
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
#Set source address on outgoing packet eth1
-A POSTROUTING -o eth1 -j SNAT --to-source 111.111.111.52
COMMIT
# Completed on Sat Dec 12 11:02:07 2009
# Generated by iptables-save v1.4.4 on Sat Dec 12 11:02:07 2009
*filter
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
#log incomming and outging traffic on eth1
-A OUTPUT -o eth1 -j LOG
-A INPUT -i eth1 -j LOG
COMMIT
# Completed on Sat Dec 12 11:02:07 2009

ip ru sh
0:      from all lookup local
32765:  from all fwmark 0x10 lookup mail
32766:  from all lookup main
32767:  from all lookup default

ip ro sh table mail
default via 111.111.111.49 dev eth1

Ok, thats looks pretty complicated. Anyway what I understood (I did some
guessing about where your second ISP connection would be): You have
three networks:
- your local network (network 192.168.1.0/24 with NAT gateway
192.168.1.3 (your server) to ISP1 and with NAT gateway 192.168.1.15 to ISP2)
- ISP1 for smtp (network 111.111.111.32/29 with gateway 111.111.111.49)
- ISP2 for the rest (network unknown with gateway unknown)

And this is what you want:
- All traffic from the local network is routed to your server (default
gateway for hosts is 192.168.1.3).
- Your server (192.168.1.3 and 111.111.111.52) has two routing tables,
all traffic is routed by the default table except for port 25 traffic
(which is marked with 0x10) which is routed by the 'mail' routing table.
- Iptables on your server applies NAT on your mail traffic, and the mail
routing table sends traffic to ISP1 (via 111.111.111.49).
- All other traffic is routed by the default routing table to
192.168.1.15 which will presumably perform NAT by itself and send it to
ISP2 (via a unknown gateway).

Is this correct?

Yes, you have this correct.


I don't understand this rule:
#mark outgoing tcp/25 packets for routing
-A OUTPUT -p tcp -m tcp --sport 25 -j MARK --set-mark 0x10


You are correct yes, however it is also our main mail gateway, ie it
revcieves mail as well, that is what the -sport is for.

I just noticed where there may be a flaw, I will change and get back
to you (after I ate lunch :) )

Ok after rethinking my strategy, I made some changes and now I am
logging packets, and this is what I saw

Dec 12 14:05:50 proxy kernel: [11676.519480] IN= OUT=eth0
SRC=192.168.1.3 DST=196.43.0.142 LEN=60 TOS=0x10 PREC=0x00 TTL=64
ID=60433 DF PROTO=TCP SPT=54928 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
MARK=0x10

Notice that it is marked, but still gets routed out through eth0....

I think my routing table is flawed :(

Regards
Ian


You are marking all outgoing traffic with source port 25, but smtp
traffic has destination port 25. Shouldn't this rule read:
-A OUTPUT -p tcp -m tcp --dport 25 -j MARK --set-mark 0x10
or even better:
-A OUTPUT -p tcp -m tcp -i eth0 --dport 25 -j MARK --set-mark 0x10

If the above description is correct, then I think that the "-A
POSTROUTING -o eth1 -j SNAT --to-source 111.111.111.52" rule is breaking

It is necessary so that the other end know what my IP is for back
routing, else it sends all packet as if from 192.168.1.3.

stuff. It is NATing *all* traffic (where it should only NAT port 25
traffic). Possibly 192.168.1.15 will only NAT traffic from the
192.168.1.0/24 subnet, but because of the iptables rule, traffic will
come from 111.111.111.52, which is thus ignored. This is remedied by
changing the rule to:
-A POSTROUTING -o eth1 -j SNAT --dport 25 --to-source 111.111.111.52

I hope I understood your setup correctly, and this message is of any
help ;)

BTW, for debugging these kind of problems, tcpdump and/or wireshark can
be of great help.

Werner

--
ubuntu-users mailing list
ubuntu-users@xxxxxxxxxxxxxxxx
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users



--
ubuntu-users mailing list
ubuntu-users@xxxxxxxxxxxxxxxx
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Relevant Pages

  • Re: Dual NICs, Routing Problem
    ... There can only be one default gateway (unless you are using extra stuff as ... network that is NOT directly attached to one of your interfaces. ... >> Do all the hosts on 192.1.36.0 know that if they want to talk to any host on ... Here, the web server needs to know the IP of eth0 on the DataBase Server, ...
    (alt.os.linux.suse)
  • Re: DNS but No Web
    ... Routng how toTo setup Windows 2000/2003 as a router for a LAN, you need to two network adapters. ... To enable LAN routing. ... the server IP routing is disable that is the problem. ...
    (microsoft.public.windows.server.networking)
  • Re: AIX 1.3 Failures and Fables
    ... DESTINATION GATEWAY FLGS REFCNT USE INTERFACE ... Is my interpretation of the AIX 1.3 #man route correct? ... Manually manipulates the routing tables. ... Is the destination host or network. ...
    (comp.sys.ibm.ps2.hardware)
  • Re: Default Gateway on VPN subnets
    ... Now I understand that you are using the TZ170 VPN routers to establish the ... Once the default gateway of the remote 2003 server was automatically ... to access the internet or the SBS network? ...
    (microsoft.public.windows.server.sbs)
  • Re: AIX 1.3 Failures and Fables
    ... I don't know if 1.3 has SMIT or its equivalent but there should be some way to add a default gateway which is your router. ... Is my interpretation of the AIX 1.3 #man route correct? ... Manually manipulates the routing tables. ... Is the destination host or network. ...
    (comp.sys.ibm.ps2.hardware)