Re: [PATCH 1/1] lro: Generic Large Receive Offload for TCP traffic

Hi Jörn

On Friday 03 August 2007 15:41, Jörn Engel wrote:
On Fri, 3 August 2007 14:41:19 +0200, Jan-Bernd Themann wrote:

This patch provides generic Large Receive Offload (LRO) functionality
for IPv4/TCP traffic.

LRO combines received tcp packets to a single larger tcp packet and
passes them then to the network stack in order to increase performance
(throughput). The interface supports two modes: Drivers can either pass
SKBs or fragment lists to the LRO engine.

Maybe this is a stupid question, but why is LRO done at the device
driver level?

If it is a unversal performance benefit, I would have expected it to be
done generically, i.e. have all packets moved into network layer pass
through LRO instead.

The driver seems to be the right place:
- There is the "page mode" interface that accepts fragment lists instead of
SKBs and does generate SKBs only in the end (see Andrew Gallatins
mails where he described the advantages of this approach)

- Some drivers (in particular for 10G NICs which actually could benefit
from LRO) have multiple HW receive queues that do some sort of sorting,
thus using one lro_mgr per queue increases the likelyhood of beeing able
to do efficient LRO.


+void lro_flush_pkt(struct net_lro_mgr *lro_mgr,
+ struct iphdr *iph, struct tcphdr *tcph);

In particular this bit looks like it should be driven by a timeout,
which would be settable via /proc/sys/net/core/lro_timeout or similar.

No, this function is needed for "page mode" as some HW provides
extra handling for small packets where packets are not stored in preallocated
pages but in extra queues. Thus the driver needs a way to flush old sessions
for this connection and handle these packets in a different way (for example
create a SKB and copy the data there).

Timeouts are not used at all. Experiments showed that flushing at the end
of a NAPI poll round seems to be sufficient (see Andrew's test results)
and does not affect the latency too badly.

Regards,
Jan-Bernd
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: driver packet coalesce
    ... I guess it would only be able to coalesce TCP packets that are adjacent in the same session. ... are a number of obvious and non-obvious interaction issues with LRO ... The LRO implementation (in the driver) has to be aware of all these ... issues and how the TCP stack treats them. ...
    (freebsd-net)
  • Re: driver packet coalesce
    ... And if a driver were to coalesce ... >>> packets and send something up the stack that violates mss ... No, LRO. ...
    (freebsd-net)
  • Re: [PATCH 0/4][RFC] lro: Generic Large Receive Offload for TCP traffic
    ... packets but does not modify ETH protocol ... packets used anymore. ... cases where the driver uses multiple lro_mgrs for different receive queues. ... LRO, written up in the Documentation subdirectory. ...
    (Linux-Kernel)
  • [RFC 0/3] lro: Generic Large Receive Offload for TCP traffic
    ... After some discussions on the mailing list concerning our LRO approach, ... A driver has to declare a "LRO Management" struct and a LRO descriptor array ... To pass packets to the network stack using LRO the following functions are used ...
    (Linux-Kernel)
  • Re: [PATCH 7/7] cxgb3 - Add SW LRO support
    ... the available API for a driver to pass packets up is ... LRO in the core networking code would require adding methods to pass up data segments in addition to skbs. ... The API should also allow for the driver to indicate such info. ...
    (Linux-Kernel)