Re: dma to user space

From: William D. Waddington (william.waddington_at_beezmo.com)
Date: 09/08/04 

Date: Wed, 08 Sep 2004 14:11:20 GMT

On Tue, 07 Sep 2004 19:58:27 -0500, John Fusco <fusco_john@yahoo.com> wrote:

>I have a device that generates a continuous stream of data at a rate
>greater than 300 MB/s which I need to save to disk.
>
>I would like to DMA this data to user space so that it can be saved to
>disk with O_DIRECT and eliminate any extra copies. It seems in 2.4 I
>could do this with kiobufs, but these are no longer available in 2.6.
>
>On the surface it seems simple - in user space I allocate memory with
>posix_memalign, then lock the memory with mlock(). Then all I need is
>the physical addresses so I can create a scatter gather list for DMA.
>
>All the 2.6 kernel documentation seems to imply that you must allocate
>DMA memory in the kernel. This won't work for me because I need lots of
>memory (>1GB), and I can't afford to copy data out of the kernel.
>
>Does anyone know how to do this?

Take a look at "Driver porting: Zero-copy user-space access: at
http://lwn.net/Articles/28548/ You may find the rest of the 2.6
driver porting info useful as well: http://lwn.net/Articles/driver-porting/

The zero-copy stuff is basically the kiobuf code stripped down to the bare
essentials. I have ported a couple of kiobuf-using drivers this way. Because
my devices are 32-bit DMA capable, I fall back to a kernel copy buffer if the
kernel is configured for >4G of RAM.

You can look at my code at
http://www.tahomatech.com/downloads/drivers/linux_2.6/pci/x86/compressed_tarfiles/
but there is a lot of cruft there left over from earlier versions. The basics
are in the 2.6 tree in drivers/scsi/st.c. Look for sgl_map_user_pages.

Bill 

-- 
--------------------------------------------
William D Waddington
Bainbridge Island, WA, USA
william.waddington@beezmo.com
--------------------------------------------
"Even bugs...are unexpected signposts on
the long road of creativity..." - Ken Burtch

Relevant Pages

  • Re: size_t or int for malloc-type functions?
    ... Some go as high as 3GB with special boot ... RedHat has a special Linux kernel that gives just under 4GB of user address space; a bit of kernel space is still required to keep syscalls working, ... It's mainly used by database folks, who should be moving to AMD64 now anyways (with its 2^51 bytes of user space, currently). ... of like the old extended/expanded memory hacks in the DOS days. ...
    (comp.lang.c)
  • Re: [swsusp] separate snapshot functionality to separate file
    ... >> Split does not prevent you from doing the cleanups. ... Kernel needs to use get_user, ... > memory directly from the used space. ... > not be visible to the user space at all. ...
    (Linux-Kernel)
  • Re: Virtual memory - user space kernel space
    ... It depends upon the total size of kernel code and static data. ... 1G/3G division in virutal memory is nothing but a set of pointers to ... User space pointers has to go via MMU to get the location in physical ... you bumble virtual and physical memory. ...
    (comp.os.linux.development.system)
  • Re: Huge pages and small pages. . .
    ... Since the CPU uses virtual memory always, ... The actual allocation only occurs when an access happens. ... the kernel just marks a promised ... DMA operation occurs that crosses page boundaries. ...
    (Linux-Kernel)
  • Re: user_to_phys() without mmap?
    ... |> |> |> into user space is causing large latencies and unsightly artifacts. ... |> |> |> fact being copied into kernel space. ... the hardware when that console was switched to be the one displayed. ...
    (comp.os.linux.development.system)