Re: [PATCH/RFC] A method for clearing out page cache

From: Paul Jackson (pj_at_sgi.com)
Date: 02/22/05

  • Next message: Jan Blunck: "Re: [RFC] pdirops: vfs patch" 
    Date:	Tue, 22 Feb 2005 03:26:33 -0800
To: Ingo Molnar <mingo@elte.hu>

    Ingo wrote:
    > app designers very frequently think that the VM gets its act wrong (most
    > of the time for the wrong reasons),

    As Martin wrote, when he submitted this patch:
    > The motivation for this patch is for setting up High Performance
    > Computing jobs, where initial memory placement is very important to
    > overall performance.

    Any left over cache is wrong, for this situation. The only right
    answer, no fault of the VM that it can't predict such, is to clear the
    past cache and ensure that all allocations are satisfied with node-local
    memory, and no page out delays, for all the threads in such tightly
    coupled jobs. These jobs have been sized to use every ounce of CPU and
    Memory from sometimes hundreds of nodes, and for hours or days, using
    tightly coupled MPI and OpenMP codes. Any misplaced pages (off the
    local node) or paging delays quickly leads to erratic and reduced
    performance.

    Flushing all the cache like this hurts any normal load that has any
    continuity of working set, and such flushing is not cheap. I have not
    observed much interest in doing this, outside of appropriate use when
    starting up a big HPC app, as described above, or the test and debug
    situations that you mention. For certain HPC apps, it can be essential
    to repeatable job performance.

    Granted, this might not be for most systems. Perhaps a CONFIG option,
    so that by default this worked on builds for big honkin numa boxes, but
    was an -ENOSYS error on ordinary sized systems? Though I prefer not to
    create artificial distinctions between configurations, without good
    reason, perhaps this is such a reason.

    Making the API ugly won't reduce its use much, rather just increase code
    maintenance costs a bit, and breed a few more bugs. Those who think
    they want this will find a way to do it. If something's worth doing,
    it's worth doing cleanly. 

    -- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.650.933.1373, 1.925.600.0401
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
  • Next message: Jan Blunck: "Re: [RFC] pdirops: vfs patch"

    Relevant Pages

    • Re: Opinion wanted
      ... the details including the nature of my own app (which you mischaraterized ... correct the current control (provided that control is invalid but invalid ... causes you to validate controls that have already been ... largely discreditted for good reason and has very little use these days. ...
      (microsoft.public.dotnet.framework.windowsforms)
    • Re: edit & continue
      ... > and just run the app in the dubugger to test it. ... there is little realistic way to perform unit tests, ... > solid reason, "If I encounter a simple mistake I can fix it without ... > practice talk that only seems to apply if you are following a certain ...
      (microsoft.public.dotnet.languages.csharp)
    • Re: charitable donations (was Re: WGA Strike 90%+ vote to strike)
      ... (even if you can't count cash or be trusted with money -- One gas ... any number of otherwise no-education-required jobs, ... I met one a few years ago who was begging for coffee money, ... Maybe he had a reason, maybe not, but there are so many positions ...
      (rec.arts.sf.tv.babylon5.moderated)
    • Re: Tablets? - OT
      ... - the main reason for porting was so idiots could maintain the code ... on a yellow legal pad. ... to find anyone to pay me to play with it. ... and an app that calculated the temperature from cricket chirps. ...
      (microsoft.public.vb.general.discussion)
    • Re: Why choose Kerio instead of ZA?
      ... Why would KPF2 have those open? ... > i/o is buffered, so it doesn't matter much anyway. ... but there must be some reason for it. ... It enables you to restart the app without rebooting on most ...
      (comp.security.firewalls)