Re: is there a user mode way to flush disk cache

From: JosephKK (joseph2k_at_lanset.com)
Date: 11/03/05 

Date: Thu, 03 Nov 2005 04:33:15 GMT

Peter T. Breuer wrote:

> Eric Taylor <et2@rocketship1.com> wrote:
>
>
>> "Peter T. Breuer" wrote:
>
>>> > If the cache is not emptied, then the next series of file writes will
>>>
>>> There is no way to empty cache, except by filling buffers.
>>>
>>> > not complete quickly as the system will hang my process while it
>>> > does some cache flushing operations.
>>>
>>> There is no such thing as a "cache flushing operation". I think you
>>> have buffer and cache mixed up!
>
>> You are correct here. To me, the cache is the memory that top
>> says is being cached. I don't really know how this translates
>> into buffers, although the number that top says: buff for seems to
>> move up/down with the value of cached.
>
> Buffers contain data that is being written to disk but has not yet
> arrived on the disk. Cache contains data that is already present on
> disk, thus might be data that has either been read from or written
> to disk.
>
> If a buffer is written to disk, then it becomes cache (and can be
> usurped).
>
>
>> I have not been able to find a tutorial on how disk file caching
>> works on linux, do you know of a link?
>
> It "works" in the obvious way. The only question is what imanagement
> strategy is used. For that you have the bdflush controls on buffer
> aging and so on. I suspect there is a text on memory tuning in the
> Documentation directory of the kernel source.
>
>> But what I have found, through some testing (just writing a series
>> of 2 gig files and timing how long it takes) that the behavior on
>> 2.4 vs 2.6 seems different.
>
> It is - they have different memory managers (at least, pre 2.4.10).
> The older memory manager is predictive (the control circuit uses at
> least first derivative info), and the newer is not (it's essentually
> stop-start, bang-bang or whatever one calls a square step response
> function).
>
>> On the 2.4 system, the writes go fast and the cache and number of
>> buff (in top) go up quickly until the amount of free mem is near zero.
>
>> On the 2.6 system, things begin to bog down when there's still 2 gigs
>> of free memory, and the cached value never gets more than 8gig on
>> a 12 gig system (while 2 gig are still free).
>
> You are misinterpreting what you see, I think. It may well be that
> newer kernels have params set differently in terms of being more or
> less aggressive about caching versus buffering versus swapping etc,
> but that's not the fundamental thing. One can set those numbers any
> way one likes on both.
>
> Peter
Could you give me a pointer to some documentation on how to set these
parameters? I have never been able to figure out where it is. 

-- 
JosephKK

Relevant Pages

  • Re: Discovering variable types...
    ... >memory it points to is on the heap. ... sequentially reading data, if one is randomly reading records, then a ... >project is what's prompting me to improve disk access. ... from a memory buffer I can do it in about a second. ...
    (comp.lang.pascal.delphi.misc)
  • Re: What can I check to fix system performance?
    ... it seems you have plenty of memory available: ... copies of files you have read of written lately, in a cache, in case ... processes per CPU, or 40 in all. ... Consider the disk structure. ...
    (comp.os.linux.setup)
  • Re: Caching control
    ... |> | invalidate/unmap them in order to discard the data from memory. ... |> writing out to disk. ... | easy to discard as clean disk cache. ... stating that a specific amount of RAM can be used only for I/O ...
    (comp.os.linux.development.system)
  • Re: Naive questions about Core i7 SMT
    ... Once the cache is full then obviously it becomes a synchronous throttle. ... However portions of file create operations are disk synchronous ... The problem with unzip is that you usually have a _lot_ of smallish files, and that would work better if you did sequential writing only, i.e. buffer full output files and write them one by one. ...
    (comp.arch)
  • Re: Scheduler: Process priority fed back to parent?
    ... > interactivity cache could estimate interactivity over a period of hours ... Then you don't even have to write it to the filesystem. ... For those of us with enough memory or a large variety of programs, ... That way the file is already in disk cache or on its way when the ...
    (Linux-Kernel)