Re: [PATCH v2] tcp: splice as many packets as possible at once



On Wed, Feb 04, 2009 at 01:01:46AM -0800, David Miller wrote:
From: Herbert Xu <herbert@xxxxxxxxxxxxxxxxxxx>
Date: Wed, 4 Feb 2009 19:59:07 +1100

On Wed, Feb 04, 2009 at 09:54:32AM +0100, Willy Tarreau wrote:

My server is running 2.4 :-), but I observed the same issues with older
2.6 as well. I can certainly imagine that things have changed a lot since,
but the initial point remains : jumbo frames are expensive to deal with,
and with recent NICs and drivers, we might get close performance for
little additional cost. After all, initial justification for jumbo frames
was the devastating interrupt rate and all NICs coalesce interrupts now.

This is total crap! Jumbo frames are way better than any of the
hacks (such as GSO) that people have come up with to get around it.
The only reason we are not using it as much is because of this
nasty thing called the Internet.

Completely agreed.

If Jumbo frames are slower, it is NOT some fundamental issue. It is
rather due to some misdesign of the hardware or it's driver.

Agreed we can't use them *because* of the internet, but this
limitation has forced hardware designers to find valid alternatives.
For instance, having the ability to reach 10 Gbps with 1500 bytes
frames on myri10ge with a low CPU usage is a real achievement. This
is "only" 800 kpps after all.

And the arbitrary choice of 9k for jumbo frames was total crap too.
It's clear that no hardware designer was involved in the process.
They have to stuff 16kB of RAM on a NIC to use only 9. And we need
to allocate 3 pages for slightly more than 2. 7.5 kB would have been
better in this regard.

I still find it nice to lower CPU usage with frames larger than 1500,
but given the fact that this is rarely used (even in datacenters), I
think our efforts should concentrate on where the real users are, ie
<1500.

Regards,
Willy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



Relevant Pages

  • Re: [PATCH v2] tcp: splice as many packets as possible at once
    ... initial justification for jumbo frames ... limitation has forced hardware designers to find valid alternatives. ... 4080 byte MTU: ... The performance impact is even more pronounced on a large RTT path ...
    (Linux-Kernel)
  • Re: Jumbo Frames
    ... If NIC is available for Jumbo Frames, we can try to find a tools to test. ... Microsoft Online Partner Support ... The NIC is supported (I can adjust Jumbo Frames from the NICs control ... Windows Server 2003 supports Jumbo Frame for sure but it also ...
    (microsoft.public.windows.server.networking)
  • Re: Jumbo Frames
    ... The NIC is supported (I can adjust Jumbo Frames from the NICs control ... the NIC in use is support Jumbo Frame and if it is enabled. ... Second, when you said Gear, you mean NIC or Switch? ...
    (microsoft.public.windows.server.networking)
  • Re: PCMCIA GigE adapters with jumbo frames
    ... with jumbo frames; is that close enough? ... NICs on either end. ... 1056Mbps, so the bus itself should be fast ... - Rick Jones ...
    (comp.dcom.lans.ethernet)
  • Re: recommandations for a home Gigabit switch
    ... require nics that support jumbos also... ... If his NICs don't support jumbo frames already, ...
    (comp.os.linux.hardware)