Re: excessive swap-in time

On Mon, 17 Mar 2008 16:43:48 +0000 (UTC) Michel Talon <talon@xxxxxxxxxxxxxxxx> wrote:

| And beleive it or not i am old enough to have seen Linux systems
| constantly trashing under load precisely because badly chosen pages
| were evicted, while the same computer worked perfectly fine
| with FreeBSD. I was under the impression that this problem was solved
| with recent Linux kernels. The problem is not that the computer will
| not work better if i add ten times more memory and there is no paging
| activity at all, of course. But that it can work well enough, and almost
| at the same speed with a moderate and constant stream of paging, if the
| paging algorithm is good, which means keeping statistics on what pages
| are freqently used, to avoid evicting them, and avoid using big bursts
| of large paging activities (this was Linux speciality) which almost
| always induce trashing - that is you have to page in pages you have
| recently paged out, exactly the symptom the original poster described.

One possible cause is that the pages just recently brought in for the
process that has recently become active don't have enough "history" of
being used, and still look to the kernel like easy eviction targets.

What I think is needed is for the kernel to keep stats on individual VMs
as to what its working set has typically been, and use that to figure out
what set of most active processes can have their working set fully swapped
in (within a percentage of real RAM limit that can be tuned). Then these
"most active working processes" would be marked and their pages are not
to be chosen for eviction unless all other processes are gone (in which
case, it must be time to recalculate working sets or adjust the active
process list, anyway). The idea is, the most recently made active process
should not be an eviction target. When a process changes to active state,
then things need to be recalculated (which processes are active ones is
certainly not a static condition).

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / spamtrap-2008-03-17-1454@xxxxxxxx |
|------------------------------------/-------------------------------------|
.

Relevant Pages

  • Re: How to set up a Linux machine that occupies the minimum memory footprint ?
    ... > Stick some extra disks in the machine and use a striped RAID swap setup. ... That just tells him how to make paging go faster. ... > If you get the right distro, it will come with a linux kernel that already ... Most distros support them ...
    (comp.os.linux.setup)
  • Re: How to set up a Linux machine that occupies the minimum memory footprint ?
    ... > Stick some extra disks in the machine and use a striped RAID swap setup. ... That just tells him how to make paging go faster. ... > If you get the right distro, it will come with a linux kernel that already ... Most distros support them ...
    (comp.os.linux.misc)
  • Re: Virtual Storage implementation
    ... with the working set size. ... If you give a Linux guest a 4GB virtual machine, it will have very close to a ... we have with midrange Linux sysadmins, DBAs, etc., regarding this topic. ...
    (bit.listserv.ibm-main)
  • Re: Virtual Storage implementation
    ... with the working set size. ... If you give a Linux guest a 4GB virtual machine, it will have very close to a ... we have with midrange Linux sysadmins, DBAs, etc., regarding this topic. ...
    (bit.listserv.ibm-main)
  • Re: excessive swap-in time
    ... And beleive it or not i am old enough to have seen Linux systems ... at the same speed with a moderate and constant stream of paging, ... are freqently used, to avoid evicting them, and avoid using big bursts ... of large paging activities which almost ...
    (comp.os.linux.development.system)