Re: dma to user space

From: John Fusco (fusco_john_at_yahoo.com)
Date: 09/09/04 

Date: Thu, 09 Sep 2004 02:16:54 GMT

On Wed, 08 Sep 2004 17:03:03 +0000, Norm Dresner wrote:

> "John Fusco" <fusco_john@yahoo.com> wrote in message
> news:pNs%c.18602$nA6.5125@twister.rdc-kc.rr.com...
>> I have a device that generates a continuous stream of data at a rate
>> greater than 300 MB/s which I need to save to disk.
>>
>> I would like to DMA this data to user space so that it can be saved to
>> disk with O_DIRECT and eliminate any extra copies. It seems in 2.4 I
>> could do this with kiobufs, but these are no longer available in 2.6.
>>
>> On the surface it seems simple - in user space I allocate memory with
>> posix_memalign, then lock the memory with mlock(). Then all I need is
>> the physical addresses so I can create a scatter gather list for DMA.
>>
>> All the 2.6 kernel documentation seems to imply that you must allocate
>> DMA memory in the kernel. This won't work for me because I need lots of
>> memory (>1GB), and I can't afford to copy data out of the kernel.
>>
>> Does anyone know how to do this?
>
> One thing that's worked for me in 2.0.x, 2.2.x, and 2.4.x is to use the
> "mem=" capability when booting to set the top of memory that Linux is
> allowed to use well below the top of physical memory and then allow my
> drivers and user programs to use that as shared memory. I don't see why it
> wouldn't work in 2.6.x as well. One real side benefit is that it's
> physically contiguous space.
>
> Norm

That's what I am doing now, but I have found that this memory is currently
not compatible with O_DIRECT. If you try to write to any device using
O_DIRECT from this memory, the process gets stuck in an infinite loop of
"minor" page faults and cannot be killed. I can post a very simple
example if anyone is interested.

Thanks.

John

Relevant Pages

  • Re: PCIe device driver question
    ... I was thinking that the MMIO and reserve memory ... pages, allocated several additional memaligned pages in user space, used ... kernel using an ioctlcall, ... The DMA was just NOT happening on ...
    (Linux-Kernel)
  • Re: PCIe device driver question
    ... I was thinking that the MMIO and reserve memory ... pages, allocated several additional memaligned pages in user space, used ... kernel using an ioctlcall, ... The DMA was just NOT happening on ...
    (Linux-Kernel)
  • RE: How to make mmaped kernel buffer non-cacheable
    ... I have mapped some user space memory to the kernel buffer and I wish to ensure that the contents of both are coherent and correctly ordered. ... The dma mapping API provide the neccessary ...
    (Linux-Kernel)
  • Re: dma to user space
    ... >I would like to DMA this data to user space so that it can be saved to ... then lock the memory with mlock. ... >the physical addresses so I can create a scatter gather list for DMA. ... >DMA memory in the kernel. ...
    (comp.os.linux.development.system)
  • [PATCH] spelling fixes: arch/arm/
    ... copy data to/from buffers located outside the DMA region. ... the remainder of memory is at the top and the DMA memory ... Peers on the PCI bus running ... - * devices are controlles by different NPUs on the same bus, ...
    (Linux-Kernel)