Re: [PATCH 3/4] myri10ge - Driver core

Anton Blanchard wrote:
+/*
+ * Set of routunes to get a new receive buffer. Any buffer which
+ * crosses a 4KB boundary must start on a 4KB boundary due to PCIe
+ * wdma restrictions. We also try to align any smaller allocation to
+ * at least a 16 byte boundary for efficiency. We assume the linux
+ * memory allocator works by powers of 2, and will not return memory
+ * smaller than 2KB which crosses a 4KB boundary. If it does, we fall
+ * back to allocating 2x as much space as required.
+ *
+ * We intend to replace large (>4KB) skb allocations by using
+ * pages directly and building a fraglist in the near future.
+ */


You go to a lot of trouble to align things. One thing on ppc64 is that
we really want to start all DMA writes on a cacheline boundary. We
enforce that in network drivers by making NET_IP_ALIGN = 0 and having
the drivers do:

skb_reserve(skb, NET_IP_ALIGN);

It sounds like your small allocations will be only aligned to 16 bytes.


We didn't get any ppc64 with PCI-E to run Linux so far. What performance
drop should we expect with our current code ?

Id suggest using the dma API instead of the pci API. We have seen
machines in the field that have failed large pci_alloc_consistent calls
because it always asks for GFP_ATOMIC memory (it presumes the worst).
The dma API allows you to pass a GFP_ flag in which will have a much
better chance of succeeding when you dont need GFP_ATOMIC memory.


Good idea, thanks,

+#ifdef CONFIG_MTRR
+ mgp->mtrr = mtrr_add(mgp->iomem_base, mgp->board_span,
+ MTRR_TYPE_WRCOMB, 1);
+#endif

...

+ mgp->sram = ioremap(mgp->iomem_base, mgp->board_span);


Not sure how we are meant to specify write through in drivers. Any ideas Ben?


I am not sure what you mean.
The only ppc64 with PCI-E that we have seen so far (a G5) couldn't do
write combining according to Apple.

Thanks,
Brice

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: Problem with linked list
    ... !pool and see what piece of memory has ... allocated (thankfully you tag your allocations). ... > The buffer is filled the following way: ... >> what's in the section of MYSTRUCT (which you don't appear to have ...
    (microsoft.public.development.device.drivers)
  • Re: vfprintf() has a 4096-byte memory leak?
    ... />>/be a memory leak in vfprintf. ... > This is probably the buffer which stdio uses for all I/O. ... test1 memory debug output: ... 1059825156: 1: top 10 allocations: ...
    (freebsd-current)
  • Re: Questions about memory management
    ... length line from a file using dynamic memory so as not to have any ... resulting from buffer overrun could result in your program executing ... program it frees any unfreed memory and destroys any allocations that ... compiler is obliged to assume that exit returns an int. ...
    (comp.lang.c.moderated)
  • Re: Memory alloc/dealloc performance
    ... > memory management tutorials? ... vectorbuffer; ... By removing memory allocations from inner loops, ... allocations using some kind of memory pool. ...
    (microsoft.public.vc.language)
  • RE: 2003 SBS stalling randomly
    ... A memory leak occurs in an application using the Volume Shadow Copy Service ... Poolmon displays data that the ... The data is grouped by pool allocation tag. ... Press P twice to display allocations from only the paged pool. ...
    (microsoft.public.windows.server.sbs)