Re: [ofa-general] Re: Demand paging for memory regions

On Tue, Feb 12, 2008 at 02:41:48PM -0800, Roland Dreier wrote:
> > Chelsio's T3 HW doesn't support this.

> Not so far I guess but it could be equipped with these features right?

I don't know anything about the T3 internals, but it's not clear that
you could do this without a new chip design in general. Lot's of RDMA
devices were designed expecting that when a packet arrives, the HW can
look up the bus address for a given memory region/offset and place
the

Well, certainly today the memfree IB devices store the page tables in
host memory so they are already designed to hang onto packets during
the page lookup over PCIE, adding in faulting makes this time
larger.

But this is not a good thing at all, IB's congestion model is based on
the notion that end ports can always accept packets without making
input contigent on output. If you take a software interrupt to fill in
the page pointer then you could potentially deadlock on the
fabric. For example using this mechanism to allow swap-in of RDMA target
pages and then putting the storage over IB would be deadlock
prone. Even without deadlock slowing down the input path will cause
network congestion and poor performance for other nodes. It is not a
desirable thing to do..

I expect that iwarp running over flow controlled ethernet has similar
kinds of problems for similar reasons..

In general the best I think you can hope for with RDMA hardware is
page migration using some atomic operations with the adaptor and a cpu
page copy with retry sort of scheme - but is pure page migration
interesting at all?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: Prevents Events From Interrupting Events
    ... download. ... Since your design already has a deadlock condition as it is, I would be surprised if you can fix that basic design issue by adding a new thread and you could easily just make it harder to fix. ... you can use the lockstatement or similar synchronization mechanism multiple times without any trouble. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Linux 2.6.9 pktgen module causes INIT process respawning and sickness
    ... poor design of dev_queue_xmit or the driver layer for xmit packets. ... receive data off the card at high enough rates. ... Linux with a Spirent Smartbits, ...
    (Linux-Kernel)
  • Re: a Christmas day compressor
    ... I have not finished writing this. ... This design is a gift in every sense, I promise never to attempt to ...  In my toy compressor corrections are necessary because the ... This design is based on the principle that information sent in packets ...
    (comp.compression)
  • a Christmas day compressor
    ... I am giving this away in the spirit of Christmas, ... haven't been forgiving and friendly. ... This design is a gift in every sense, I promise never to attempt to ... This design is based on the principle that information sent in packets ...
    (comp.compression)
  • Re: [PATCH] 1/2 Maple: Update bus driver to allow support of VMU device
    ... A trylock is always an exceptional thing. ... If I used down then a deadlock is a near certainty. ... queue of packets, one the queue of output packets. ... What the heck is that trying to do?!?!?! ...
    (Linux-Kernel)