Re: Caching control

On Tue, 10 Mar 2009 16:05:47 -0700 (PDT) David Schwartz <davids@xxxxxxxxxxxxx> wrote:
| On Mar 10, 2:40?pm, phil-news-nos...@xxxxxxxx wrote:
|
|> If posix_fadvise(POSIX_FADV_NOREUSE) really would tell the kernel that what this
|> descriptor is writing will not need to be read back anytime soon (and more will
|> be written after it than can even be held in RAM at all), then that should do it.
|
| That's precisely what it does.
|
|> But it seems POSIX_FADV_NOREUSE really doesn't do anything in Linux (which is
|> still compliant, as this is advisory).
|
| Because nothing should be necessary.

Since it is keeping the pages in RAM after being written, this doesn't seem
to be consistent.


| Your issue is that for some reason, large sequential writes are
| forcing more swapping than they should. This is a normal use case and
| the system is supposed to handle it sanely. The question is, why is
| this causing you a problem when it doesn't cause other people
| problems?

What it does could be considered sane with respect to the speculation that
the data being written, being newer, is more likely to be read back in the
near future, than data that was mapped in by some other process a long time
ago (like many seconds to a minute).


| IMO, the five most likely problems are:
|
| 1) You have too little physical memory.

I have way way more than is needed.


| 2) You have too little swap.

Ironically, the probless is less severe, the less swap there is.


| 3) You've tweaked some setting that's affecting the system's swapiness
| level.

I didn't tweak any settings on a "persistent across reboots" way that I am
aware of.


| 4) You have a version of the Linux kernel with a bug or regression
| that causes you a problem.

Multitutes of 2.6 kernel versions. I'm currently on 2.6.26.2.


| 5) Your use case is so weird the kernel mishandles it.
|
| What kernel version are you using? How much physical RAM do you have?
| How much swap do you have? Do you have an estimate for your working
| set size (just counting direct memory usage, not counting disk cache)?

version 2.6.26.2 currently
RAM 8 GB
swap partition 2 GB
swap activated 0 GB (works best this way)
working set 1 to 2 GB est.

--
|WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
| by the abuse department, bellsouth.net is blocked. If you post to |
| Usenet from these places, find another Usenet provider ASAP. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |
.

Relevant Pages

  • oom-killer: gfp_mask=0xd1 with 2.6.12 on EM64T
    ... On a dual EM64T Xeon with 4GB of RAM, ... all swap space still available). ... oom-killer kills one or two of those processes, with the kernel messages ...
    (Linux-Kernel)
  • Re: Simple script that locks up my box with recent kernels
    ... >> and just run the resulting kernel version for a day or two. ... then I'll restart bisection and give each ... The box has 2GB of RAM and 768MB swap. ...
    (Linux-Kernel)
  • Re: Caching control
    ... You have too little swap. ... Your use case is so weird the kernel mishandles it. ... How much physical RAM do you have? ... set size (just counting direct memory usage, ...
    (comp.os.linux.development.system)
  • Re: running Linux with no swap space (but lots of RAM)
    ... | expense of RAM. ... what would have to be running to make use of 256 GB of swap? ... As long as there is still unallocated memory, ... At this point, the kernel is ...
    (comp.os.linux.development.system)
  • Re: Fedora Core 2 - Processes Eating 20X more memory
    ... will see this happen because FC2 uses the 2.6 kernel, ... Overall it is designed to use more ram and less swap. ... upgraded from FC1 to FC2. ...
    (alt.os.linux.redhat)