Re: RFC: ipath ioctls and their replacements

Eric> Roland you know the RDMA model best, are things so tied to
Eric> the current crop of infiniband protocols that what the ipath
Eric> code wants to do is not covered?

Eric> They clearly need subsystem support and what they are trying
Eric> to do either isn't covered or they don't see how to use what
Eric> is there. Do the infiniband verbs not allow dealing with a
Eric> unreliable datagram protocol?

I think this has been answered already but the issue is really that
the PathScale hardware does not implement RDMA or even any of the
other connection-oriented abstractions that the RDMA layer is designed
for. The hardware has only much lower level capabilities, which
basically can send and receive packets on an IB link.

With those capabilites it is possible to implement IB transports in
software -- so for example RDMA read operations are simulated by
having the CPU on the receiver copy data to send the response.
However that implementation is not going to make good use of the IB
midlayer, which really operates at the abstraction level above the IB
transport.

It's also possible to use the PathScale hardware to directly implement
MPI on top of a protocol optimized specifically for MPI, without using
IB verbs semantics or an IB transport on the wire. But clearly the
userspace interface needed for doing this is not going to match up
very well with a userspace interface for IB verbs (which is at a
different abstraction level).

- R.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: [ofa-general] Re: Demand paging for memory regions
    ... The remote side can send RDMA WRITE traffic at any time after ... There is no way to signal that a page is not ready and the remote ... motivation for this is kernel bypass where the user space app wants ... The wire protocol consists of packets that say 'Write XXX bytes to ...
    (Linux-Kernel)
  • Re: [ofa-general] Re: Demand paging for memory regions
    ... significant change to the general RDMA operation and requires changes to ... both sides of the connection and the wire protocol. ... Yes it may require a separate connection between both sides where a ... RDMA protocol and get some support from some of the hardware - but it ...
    (Linux-Kernel)
  • Re: [ofa-general] Re: Demand paging for memory regions
    ... significant change to the general RDMA operation and requires changes to ... both sides of the connection and the wire protocol. ... allows a notification to the other side that the mapping is being down ... for access control and enforcing the liftime of the mapping. ...
    (Linux-Kernel)
  • Re: [PATCH 01/03] 9prdma: RDMA Transport Support for 9P
    ... > This file implements the RDMA transport provider for 9P. ... And actually, I don'treally care why it stopped waiting, I only care about the state of the connection and if it's not what I need it to be I error out. ... I suppose there is a theoretical race here where the user cancels the wait with ctrl-C, but the connection keeps going because the transport always gets back fast enough. ... There is the potential for RDMA ops in the future. ...
    (Linux-Kernel)
  • [PATCH mmotm] net/9p: fix build error
    ... (I did this patch for linux-next & posted it, but it's not merged anywhere, ... Make 9p's RDMA option depend on INET since it uses Infiniband rdma_* ... tristate "9P Virtio Transport " ...
    (Linux-Kernel)