Re: running Linux with no swap space (but lots of RAM)

On Thu, 29 Nov 2007 15:55:33 +0100 Rainer Weikusat <rweikusat@xxxxxxxxxxx> wrote:
| phil-news-nospam@xxxxxxxx writes:
|
| [...]
|
|> | e> In that situation it is quite likely that you don't have "some
|> | e> other device" to swap to. That's probably why you are booting
|> | e> from flash in the first place.
|> |
|> | The problem in your new hypothetical is then that you have no device
|> | capable of tolerating paging I/O, not that the system is paging to
|> | disc.
|>
|> Please explain what you mean by "tolerating paging I/O".
|
| In plain words, this means 'only devices whose actual behaviour and
| limitations the OP understands even less well than he believes to
| understand disks'.
|
| [...]
|
|> A frequent scenario I see happens when my need for memory by user space
|> programs is below the capacity is available, and would not have even begun
|> to swap anything. A program is run that will be doing a large amount of
|> I/O output, such as copying 100+ GB of files between filesystems. The I/O
|> buffering goes beyond just the pages that are free. The buffering logic
|> tries to buffer far more of those 100+ GB than needed to keep the writing
|> drive continuously busy or even to minimize head seeks.
|
| The purpose of the page cache is neither 'to keep the drive
| continously busy' nor 'to minimze head seeks'. Both would be tasks the
| elevator (or I/O-scheduler) is supposed to accomplish.

Some amount of caching is necessary to achieve such I/O scheduling.
More than that can improve it further. But at some point, the returns
are not worth the costs.

What _is_ true is that a process doing write() calls as fast as it can
will exceed the capacity of even the fastest I/O devices even when the
I/O is most optimal. Given such a process doing write()s, and given
the physical writing taking place slower than that, at what point should
one of those write() calls be blocked (or return EAGAIN if in non-blocking
mode)? How _much_ write caching is truly needed?

The problem is, the system doesn't really stop it at a fixed level of
usage. If there are no immediately available pages in RAM, it will block
the write() because it has no choice. But the system will proceed to
swap out process pages, in addition to continuing the I/O scheduling it
has to obviously be doing.

I want to find a way to minimize or eliminate the impact of large scale
writing on process VM page residency. Having no swap at all (but having
added more RAM in the amount of that swap) will achieve a lot of that.
Perhaps a better way is tunable parameters stating what portions of RAM
can be use only for process pages, what portions of RAM can be use only
for I/O buffering, and what portions of RAM can be used for either purpose.
With such parameters, if the amount of process pages present do not exceed
the reserved amount of RAM for them, then they cannot be stolen from to
make room for more I/O buffering. It would also be a plus to have an
fcntl() call per fd to allow a process doing the writing to further limit
their own queueing.


|> The end result is that other programs that were not immediately in
|> use (for example Firefox, Thunderbird, and even Xorg itself), get
|> substantially swapped out.
|
| 'programs' (ie text pages corresponding to running processes) are
| never 'swapped out', because the corresponding memory pages are
| usually clean, so their contents can be discarded and later re-read
| from disk.

Not all such pages are clean like that. For various reasons, CoW has
been triggered, such as address updates during run time linking.

But for the rest, yes, they can be abandoned (stolen), which forces
them to be swapped back in. Unfortunately, a swapless system will not
stop this. I WANT A WAY TO PREVENT THIS FROM HAPPENING when it is due
to I/O buffering.


| Just for an informal test, I have just created an archive of all of my
| filesystem to /dev/null (~16G). The amount of memory allocated to
| 'buffers' and 'cached' (vmstat) peaked well below 95000K and 45000K,
| respectively. No paging activities occured during creation of the
| archive.

Well of course no paging activities occured during creation of the archive.
This is a bogus test. Try _real_ writing to a _real_ device. The amount
of such writing to reach the effect depends on factors like the size of RAM,
the speed of your I/O device, etc. I don't have particulars on your system
so I can't give very good numbers. But I might suggest a much larger amount
such as dd'ing zeros to a partition to erase all sectors (without using the
direct write feature, which has its own issues).


| BTW, the by-and-far easiest path to personal happiness for you in this
| respect is to just misconfigure your system to your hearts content
| (its yours, after all) instead of talking about hypothesises you have
| about situations which - for some strange reason - are not that
| generally reproducible than the generality of your inferences would
| require.

Someone who thinks writing to /dev/null would result in lots of data being
queued for writing to the physical device is not someone I would care to
consider technical advice from.


| It would still be more sensible to add RAM until you don't experience
| regular paging activities occuring for some unknown reason on your
| system and leave the virtual memory configuration as-is to deal with
| non-regular situations. Or consider reducing your working set.

I already know that the amount of RAM to avoid the problem is too radical
to consider. And I have yet to find a mainboard that supports 2 CPUs and
2 TB of RAM and fits in an ATX case using no more than 550 watts of power.

Despite your bogus test, the reality is that even with a working set well
below the amount of actual RAM present, I/O writing (to real devices) will
gradually force out pages. A swapless system is a partial answer to this
problem. My other idea is to have even more RAM, set up a big RAMFS, and
at system initialization, copy /opt and /usr into there, and bind mount
that over the real /opt and /usr, in read-only mode. That would be another
8GB of RAM, bringing my system that would be well configured at 4GB in the
usual way, up to 16GB (4GB for the original RAM need, 4GB more for swapless
RAM space, and 8GB for /opt and /usr).

I really shouldn't have to structure the system to avoid this issue. There
should be tuning knobs that allow me to reserve some portion of RAM (not a
classification of particular RAM locations) to be used for process pages
(e.g. program and library text pages), whether copied-on-write or not, that
cannot under any circumstances ever be forced out due to lots of memory
being used for write() buffering ... even if the triggering event is a need
for another process page (it must satisified by taking a page used for the
write buffering but grabbing for its use the next one freed up be completion
of the current physical writing).

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / spamtrap-2007-11-30-0806@xxxxxxxx |
|------------------------------------/-------------------------------------|
.

Relevant Pages

  • Re: built-in file insertion routine?
    ... >I was looking to see if there was a built in routine for inserting text into ... It has to be done either by writing a new file up to the insert point, ... combing in RAM and write the thing out as in one i/o. ... See http://mindprod.com/jgloss/fileio.html for how to create the i/o. ...
    (comp.lang.java.programmer)
  • Re: Alpha remembrance day
    ... just read the 8400 configuration guide. ... impossible to configure with redundant I/O and 14CPU's and 14GB of RAM. ... You don't need 64bit support to ...
    (comp.os.vms)
  • Re: running Linux with no swap space (but lots of RAM)
    ... The buffering logic ... | into memory. ... Reading is does not create the same situation as writing out. ... By that time a huge amount of RAM is ...
    (comp.os.linux.development.system)
  • Re: efficacy of Linux w/o swap
    ... |> dedicated purpose that will use Linux and commodity hardware. ... | I suppose you could lock into RAM, ... can account for such an extreme amount of time. ... then evetually complete the switch. ...
    (comp.os.linux.development.system)
  • Re: xp pro blue screen
    ... Insufficient RAM. ... the amount of RAM, and performance seems to suffer quite badly if and ... Windows Task Manager is basically useless for this type of assessment ...
    (microsoft.public.windowsxp.help_and_support)