Re: Help - Memory Leak Issue with SCSI Stack on Linux

From: John-Paul Stewart (jpstewart_at_binaryfoundry.ca)
Date: 08/30/04 

Date: Mon, 30 Aug 2004 10:57:58 -0400

T wrote:
>
> For this particular test (running iometer on Linux system), could you give
> me some detailed information on how Linux buffer and cache are used and why
> they increase little by little?

There's nothing specific to the test to say.

Any time any application reads from disk, it requests certain blocks.
The kerenel responds by reading those blocks *plus* some more (on the
assumption that the next blocks will be needed soon) and stores those
extra blocks in the cache. If/when the application requests the
subsequent blocks, they are read from the kernel buffer which is many
times faster than reading from the physical disk.

If the application requests more blocks that aren't in the cache, the
first process (reading extra blocks) is used and the cache grows in size
to contain them. That's why the cache grows over time. Of course,
there's an upper limit placed on the size of cache based on available
memory, which is why you see it stop growing.

As to the difference between buffers and cache, AIUI one is for stuff on
the way in from disk, and the other is for stuff not yet written out to
disk. I can never remember which is which, though.

> The buffers and cache info from 'top' should be the same as those from
> /proc/meminfo, correct?

Yes. 'top' probably just reads /proc/meminfo.

> BTW, is 'free' anonther system resource monitor like 'top'?

'free' is a lot less than 'top'. It just gives memory info, and only
does it once (unlike 'top' which is usually continuously monitoring the
system state). It's standard with all Linux distros I've used. Here's
an example of the output of 'free':

            total used free shared buffers cached
Mem: 1034616 1020500 14116 0 142440 181356
-/+ buffers/cache: 696704 337912
Swap: 1048568 5428 1043140

Reading across the first line, it shows 1034616 KBytes of memory with
1020500 used and 14116 free and goes on to show 142440 KBytes of buffers
and 181356 in the cache. At first glance it looks like this system is
almost out of memory---only 14 MB out of 1GB is free. But with 142MB in
buffers and another 181MB of cache (both of which can be used for
application if they needed it), it's nowhere near out of memory. The
second line does that math for us, showing that if you neglect
buffers/cache there's really only 696704 KBytes used and 337912 KBytes free.

Relevant Pages

  • Re: Increased Buffers due to patch 56e49d (vmscan: evict use-once pages first), but why exactly?
    ... All is about the increased amount of "Buffers" accounted as active while ... But I didn't get how this prefers buffers compared to cache pages (I ... This indeed sounds like the kind of workload that would only ... filesystem metadata all the time. ...
    (Linux-Kernel)
  • Re: Increased Buffers due to patch 56e49d (vmscan: evict use-once pages first), but why exactly?
    ... All is about the increased amount of "Buffers" accounted as active while ... But I didn't get how this prefers buffers compared to cache pages (I ... This indeed sounds like the kind of workload that would only ... it might makes stream I/O benchmark score a bit because such workload ...
    (Linux-Kernel)
  • Re: ReadFileScatter and WriteFileGather
    ... Page-aligned page-size buffers can be passed down to bare metal without any ... One example is a DB server that must cache disk data without using the OS ... repeatedly requesting single pages each time. ...
    (microsoft.public.win32.programmer.kernel)
  • Re: Top - Mem: 119212k used
    ... AC>> meant "60+megs after excluding cache", ... AC>> This indicates 54360K in use, after excluding buffers and cache. ... AC> is the smallest window manager I could find. ... (/usr/bin/X11/xterm, not the 'fancy' xterms), and Mozilla. ...
    (comp.os.linux.misc)
  • Re: free (command) memory question
    ... > I just ran the free command and it shows as follows: ... Of the used memory 105MB is buffers and 213MB ... would assume Linux always keeps a little physical memory for buffers and ... cache, so I would expect it to start swapping before all of the additional ...
    (alt.os.linux.suse)