Re: async network I/O, event channels, etc

On Wed, Jul 26, 2006 at 11:10:55PM -0700, David Miller (davem@xxxxxxxxxxxxx) wrote:
From: Evgeniy Polyakov <johnpol@xxxxxxxxxxx>
Date: Wed, 26 Jul 2006 10:28:17 +0400

I have not created additional DMA memory allocation methods, like
Ulrich described in his article, so I handle it inside NAIO which
has some overhead (I posted get_user_pages() sclability graph some
time ago).

I've been thinking about this aspect, and I think it's very
interesting. Let's be clear what the ramifications of this
are first.

Using the terminology of Network Algorithmics, this is an
instance of Principle 2, "Shift computation in time".

Instead of using get_user_pages() at AIO setup, we instead map the
thing to userspace later when the user wants it. Pinning pages is a
pain because both user and kernel refer to the buffer at the same
time. We get more flexibility when the user has to map the thing
explicitly.

I.e. map skb's data to userspace? Not a good idea especially with it's
tricky lifetime and unability for userspace to inform kernel when it
finished and skb can be freed (without additional syscall).
I did it with af_tlb zero-copy sniffer (but I substitute mapped pages
with physical skb->data pages), and it was not very good.

I want us to think about how a user might want to use this. What
I anticipate is that users will want to organize a pool of AIO
buffers for themselves using this DMA interface. So the events
they are truly interested in are of a finer granularity than you
might expect. They want to know when pieces of a buffer are
available for reuse.

Ah, I see.
Well, I think preallocate some buffers and use that in AIO setup is a
plus, since in that case user does not care about when it is possible to
reuse the same buffer - when appropriate kevent is completed, that means
that provided buffer is no longer in use by kernel, and user can reuse
it.

--
Evgeniy Polyakov
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: async network I/O, event channels, etc
    ... AIO I/O routines make transfer of buffer ownership. ... while kernel "owns" the buffer for an AIO ... Or do we want to allocate at least one page for one skb? ... and actually userspace does not know which NIC receives it's data, ...
    (Linux-Kernel)
  • Re: OpenGL-based framebuffer concepts
    ... >> By implementing a framework where userspace doesn't have to know - ... display kernel panics regardless of current video mode, ... changes userspace makes to the GPU configuration. ... Track where the scanout buffer is. ...
    (Linux-Kernel)
  • Re: mmap(2)ping of pci_alloc_consistent() allocated buffers on 2.4 kernels question/help
    ... >> to the userspace application, and then when in the userspace I do ... This is a RT driver for the RT Linux (that's why the 2.4 kernel). ... And then I would like to fill a buffer via ... then calls the ioctl of the RT FIFO with the buffer number ...
    (Linux-Kernel)
  • Re: [RFC][PATCH] New message-logging API (kprint)
    ... Documentation/kprint.txt (see patch). ... I started this thread by posting an idea I had for shrinking the kernel ... >    new kprint buffer instead of the regular log buffer. ... Run dmesg through a filter in userspace using lots of regular expressions. ...
    (Linux-Kernel)
  • Re: "write()" semantics
    ... Protecting the buffer has costs. ... you had declare I/O buffers (which could be static ... Obviously you're limited by how many concurrent segments the VMM can ... Kernel copy can freely be relocated ...
    (comp.arch.embedded)