Re: [RFC] [PATCH 0/3] ioat: DMA engine support

From: Andi Kleen (ak_at_suse.de)
Date: 11/23/05

  • Next message: Russell King: "Re: overlapping resources for platform devices?" 
    Date:	Wed, 23 Nov 2005 23:30:08 +0100
To: Jeff Garzik <jgarzik@pobox.com>

    On Wed, Nov 23, 2005 at 05:06:42PM -0500, Jeff Garzik wrote:
    > IOAT is super-neat stuff.

    The main problem I see is that it'll likely only pay off when you can keep
    the queue of copies long (to amortize the cost of
    talking to an external chip). At least for the standard recvmsg
    skb->user space, user space-> skb cases these queues are
    likely short in most cases. That's because most applications
    do relatively small recvmsg or sendmsgs.

    It definitely will need a threshold under which it is disabled.
    With bad luck the threshold will be high enough that it doesn't
    help very often :/

    Longer term the right way to handle this would be likely to use
    POSIX AIO on sockets. With that interface it would be easier
    to keep long queues of data in flight, which would be best for
    the DMA engine.

    > In addition to helping speed up network RX, I would like to see how
    > possible it is to experiment with IOAT uses outside of networking.
    > Sample ideas: VM page pre-zeroing. ATA PIO data xfers (async copy to
    > static buffer, to dramatically shorten length of kmap+irqsave time).
    > Extremely large memcpy() calls.

    Another proposal was swiotlb.

    But it's not clear it's a good idea: a lot of these applications prefer to
    have the target in cache. And IOAT will force it out of cache.

    > Additionally, current IOAT is memory->memory. I would love to be able
    > to convince Intel to add transforms and checksums, to enable offload of
    > memory->transform->memory and memory->checksum->result operations like
    > sha-{1,256} hashing[1], crc32*, aes crypto, and other highly common
    > operations. All of that could be made async.

    I remember the registers in the Amiga Blitter for this and I'm
    still scared... Maybe it's better to keep it simple.

    -Andi

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Russell King: "Re: overlapping resources for platform devices?"

    Relevant Pages

    • Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regress
      ... does not include the objects that could be on those queues. ... now you can only have one "queue" thats part of the same struct. ... yes, it's what i referred to as "distributed, per node cache". ... It has SLAB memory spread out amongst nodes. ...
      (Linux-Kernel)
    • Re: [PATCH] O17int
      ... globally "down-shift" periodically) ... is to periodically check the head of all queues ... Everything I've tried along these lines has a common upside: ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: [RFC] /proc/ide/hdx/settings with ide-default pseudo-driver is a 2.6/2.7 show-stopper
      ... because I can't rely on sane behaviour if I remove queues etc, ... I keep the queue around so that to the block layer nothing disappears ... and is freed up causing race issues but to the IDE layer the right stuff ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: IO scheduler, queue depth, nr_requests
      ... >> is because your controller is sucking up all your requests. ... >> request limit patch to try in a minute. ... >queues, ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: queue_work from interrupt Real time preemption2.6.11-rc2-RT-V0.7.37-03
      ... > I'm working on a tweak to the preepmtive soft IRQ implementation using work ... > queues and I'm having problems with a BUG assert when trying to queue_work. ... off interrupts, and the spinlocks under the PREEMPT kernel, may now ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)