RE: [Lhms-devel] [PATCH 0/7] Fragmentation Avoidance V19

From: Adam Litke (agl_at_us.ibm.com)
Date: 11/07/05

  • Next message: Greg KH: "Re: typedefs and structs [was Re: [PATCH 16/42]: PCI: PCI Error reporting callbacks]" 
    To: Andy Nelson <andy@thermo.lanl.gov>
Date:	Mon, 07 Nov 2005 12:58:53 -0600

    On Sun, 2005-11-06 at 17:34 -0700, Andy Nelson wrote:
    > Hi folks,
    >
    > >Not sure how applications seamlessly can use the proposed hugetlb zone
    > >based on hugetlbfs. Depending on the programming language, it might
    > >actually need changes in libs/tools etc.
    >
    > This is my biggest worry as well. I can't recall the details
    > right now, but I have some memories of people telling me, for
    > example, that large pages on linux were not now available to
    > fortran programs period, due to lack of toolchain/lib stuff,
    > just as you note. What the reasons were/are I have no idea. I
    > do know that the Power 5 numbers I quoted a couple of days ago
    > required that the sysadmin apply some special patches to linux
    > and linking to extra library. I don't know what patches (they
    > came from ibm), but for xlf95 on Power5, the library I had to
    > link with was this one:
    >
    > -T /usr/local/lib64/elf64ppc.lbss.x
    >
    >
    > No changes were required to my code, which is what I need,
    > but codes that did not link to this library would not run on
    > a kernel that had the patches installed, and code that did
    > link with this library would not run on a kernel that didn't
    > have those patches.
    >
    > I don't know what library this is or what was in it, but I
    > cant imagine it would have been something very standard or
    > mainline, with that sort of drastic behavior. Maybe the ibm
    > folk can explain what this was about.

    Wow. It's amazing how these things spread from my little corner of the
    universe ;) What you speak of sounds dangerously close to what I've
    been working on lately. Indeed it is not standard at all yet.

    I am currently working on an new approach to what you tried. It
    requires fewer changes to the kernel and implements the special large
    page usage entirely in an LD_PRELOAD library. And on newer kernels,
    programs linked with the .x ldscript you mention above can run using all
    small pages if not enough large pages are available.

    For the curious, here's how this all works:
    1) Link the unmodified application source with a custom linker script which
    does the following:
      - Align elf segments to large page boundaries
      - Assert a non-standard Elf program header flag (PF_LINUX_HTLB)
        to signal something (see below) to use large pages.
    2) Boot a kernel that supports copy-on-write for PRIVATE hugetlb pages
    3) Use an LD_PRELOAD library which reloads the PF_LINUX_HTLB segments into
    large pages and transfers control back to the application.

    > I will ask some folks here who should know how it may work
    > on intel/amd machines about how large pages can be used
    > this coming week, when I attempt to do page size speed
    > testing for my code, as I promised before, as I promised
    > before, as I promised before.

    I have used this method on ppc64, x86, and x86_64 machines successfully.
    I'd love to see how my system works for a real-world user so if you're
    interested in trying it out I can send you the current version. 

    -- 
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
  • Next message: Greg KH: "Re: typedefs and structs [was Re: [PATCH 16/42]: PCI: PCI Error reporting callbacks]"

    Relevant Pages

    • The performance and behaviour of the anti-fragmentation related patches
      ... I've posted up patches that implement the two generally accepted approaches ... physical address space on the vanilla kernel because there is no effort made ... also that the huge page allocations always come from here as well. ... Vanilla Kernel List-base Kernel Zone-base Kernel Combined Kernel ...
      (Linux-Kernel)
    • Re: This is [Re:] How to improve the quality of the kernel[?].
      ... The -mm kernel already implements what your proposed PTS would do. ... If patch have no TS ID, ... Thus i can apply for example lguest patches and implement and test new ... How many open source projects use Bugzilla and how many use the Debian BTS? ...
      (Linux-Kernel)
    • Re: Scheduler Situation
      ... understand basic kernel operations like memory management or CPU ... I, like many people, had been using Con's patches for years and were ... Con here. ... other aspect, including offering a choice of IO schedulers, but not ...
      (Linux-Kernel)
    • Re: 2.6.X, NPTL, SCHED_FIFO and JACK
      ... improvement as AM's lowlat patches. ... patches) that the author of the premiere lowlat patches for 2.4 would ... many notable kernel developers were not particularly interested in our ... VM and disk subsystems appear to be conspiring ...
      (Linux-Kernel)
    • Re: [PATCH] delete devfs
      ... been merging patches at a rate of about 10MB/month. ... Andrew would like to see a 2.6 tree which continues to change ... In his vision of the future, the kernel.org kernel will be the most ... keeps the developers happy and gets new code out to users quicker. ...
      (Linux-Kernel)