Re: Memory Performance Issue with Fedora Core 2 Kernels

From: James Foris (james.foris_at_med.ge.com)
Date: 07/30/04

  • Next message: Christian Loza: "About floppy" 
    Date: Fri, 30 Jul 2004 16:46:58 -0500
To: For users of Fedora Core releases <fedora-list@redhat.com>

    James Wilkinson wrote:
    > James Foris wrote:
    >
    >>I have run into an issue with memory bandwidth using the Fedora Core
    >>2 kernels and I need help. I don't know what is wrong, but something
    >>killed performance of my custom driver when I ported it from RedHat 7.3
    >>to Fedora Core 2. I believe I have narrowed it down to the kernel.
    >>
    >>My driver requires a large amount of contiguous physical memory for
    >>DMA from a PCI device. I use the 'mem=YYY' command line parameter to
    >>reserve the top of physical RAM for my driver. Then I allow mapping
    >>via mmap() calls to user space. The user space app then uses this
    >>pointer to save the data to disk.
    >>
    >><snip>
    >>
    >>I have tried this on several platforms and kernels and the results vary,
    >>but the common denominator seems to be:
    >>
    >> Fedora kernel + 32-bit Intel = poor performance (see below)
    >
    >
    > I suggested:
    >
    >>Wild guess: have you tried turning off the 4G+4G support in the kernel
    >>(requires a recompile)?
    >
    >
    > James replied
    >
    >>I have rebuilt now w/o several patches; 4Kstacks, netdump, and 4g/4g.
    >>
    >>The problem has gone away. I suspect that 4G/4G is broken.
    >
    >
    > It sounds more "broken as designed", to be honest. It sounds as though
    > your setup is doing a *lot* of context switches between user mode and
    > kernel mode. The basic trade-off that 4G/4G flushes the TLB each
    > context switch, so that kernel and user both get nearly 4 GB, and TLB
    > flushes are expensive.
    >
    > See
    > http://lwn.net/Articles/39283/
    > and
    > http://kerneltrap.org/node/view/2891
    >
    > May I recommend fedora-devel-list@redhat.com or the LKML? You might not
    > have much done about it, but they might be able to recommend
    > workarounds.

    Actually, we have gotten some of the kernel group at Intel into the game, now.

    It seems that the problem can be completely reproduced using the native
    "/dev/mem" interface - no custom drivers needed. The intel engineer that
    has looked at this believes that either the 4g/4g patch is broken, or that there
    are mem driver interactions with 4g/4g that no one has taken into account
    (which to me says that potentially both the mem driver and 4g/4g patch need
    fixing).

    Certainly, removing the 4g/4g patch fixes this problem.

    We have been told that Intel will be talking/working with the Red Hat people
    directly to resolve this. They seem to think that this is something significant
    that should be resolved.

    >
    > At the very least, it would be good if real-life problems with 4G/4G are
    > reported.
    >
    > James.
    > 

    -- 
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: http://www.redhat.com/mailman/listinfo/fedora-list
  • Next message: Christian Loza: "About floppy"

    Relevant Pages

    • Re: Which to choose FC1 or FC2
      ... > so I get the Vesa driver and so no mplayer and the like. ... You should be able to get the NVidia drivers from the NVidia site, ... Please note that this will NOT work with unmodified Fedora Core 2 ... kernels due to a change in kernel stack size. ...
      (Fedora)
    • Re: 2,6.26-rc4-git2 - long pause during boot
      ... James Bottomley wrote: ... I've also built 2.6.26-rc5-git2 with verbose USB ... A boot log from that kernel is attached. ... matched device 3-2 with driver usb ...
      (Linux-Kernel)
    • Re: Memory Performance Issue with Fedora Core 2 Kernels
      ... James Wilkinson wrote: ... >>I have run into an issue with memory bandwidth using the Fedora Core ... >>2 kernels and I need help. ... >>killed performance of my custom driver when I ported it from RedHat 7.3 ...
      (Fedora)
    • Re: Memory Performance Issue with Fedora Core 2 Kernels
      ... James Foris wrote: ... > I have run into an issue with memory bandwidth using the Fedora Core ... I believe I have narrowed it down to the kernel. ... > via mmapcalls to user space. ...
      (Fedora)
    • Re: KVM and MOUSE problems - REVISED -- FILE PASTED IN
      ... > title Fedora Core ... That should do it for the latest kernel, ... James Wilkinson ...
      (Fedora)