Re: help me understand memory and processes

From: Nick Landsberg (hukolau_at_NOSPAM.att.net)
Date: 05/30/04 

Date: Sun, 30 May 2004 20:20:16 GMT

Jean-David Beyer wrote:

> peter pilsl wrote:
>
>> Jeroen Geilman wrote:
>>
>>>
>>> All instances of an executable share the *same* code in memory, but
>>> they all have their *own* data and stack space.
>>>
>>> You seem to be confusing these two.
>>>
>>> If you run top with the options I mentioned then you will see exactly
>>> how much individual (non-shared) memory each process has *claimed* -
>>> that is the closest figure you will get from anywhere.
>>>
>>> Why do you think you can't get the exact amount ?
>>> Surely the kernel will know, since it manages all memory.
>>>
>>
>> I run top to show me the DATA (=data+stack)-space, which you tell to
>> be the unshared part.
>>
>> PID USER VIRT RES SHR %MEM SWAP CODE DATA COMMAND
>> 29596 root 25632 25m 8760 2.5 0 28 24m java
>> 29603 root 25632 25m 8760 2.5 0 28 24m java
>> 29624 root 25632 25m 8760 2.5 0 28 24m java
>> 29630 root 25632 25m 8760 2.5 0 28 24m java
>> 29631 root 25632 25m 8760 2.5 0 28 24m java
>> 29633 root 25632 25m 8760 2.5 0 28 24m java
>> 29634 root 25632 25m 8760 2.5 0 28 24m java
>> 29635 root 25632 25m 8760 2.5 0 28 24m java
>> 29636 root 25632 25m 8760 2.5 0 28 24m java
>> 29637 root 25632 25m 8760 2.5 0 28 24m java
>> 29638 root 25632 25m 8760 2.5 0 28 24m java
>> 29639 root 25632 25m 8760 2.5 0 28 24m java
>> 29640 root 25632 25m 8760 2.5 0 28 24m java
>> 29641 root 25632 25m 8760 2.5 0 28 24m java
>> 29643 root 25632 25m 8760 2.5 0 28 24m java
>> 29644 root 25632 25m 8760 2.5 0 28 24m java
>> 29645 root 25632 25m 8760 2.5 0 28 24m java
>> 29653 root 25632 25m 8760 2.5 0 28 24m java
>>
>> 18 instances uses 24M DATA-space, which means to be 432M. I dont
>> believe this. I think these 18 processes share mostly all the same 24M
>> DATA.
>> I think one process starts, claims 24M Data-space and then forks
>> itself many time and this means, that the 24M are starting as shared
>> memory.

The term "shared memory" has a very specific meaning in the
Unix/Linux world. Meaning that the processes have somehow
arranged to allocate a portion of system memory and all use
it together. (See man shmget and man shmat for starters.)

What you seem to be referring to is that when a process forks
the memory pages are marked "copy on write". Until a write
happens, these pages are "shared" in the true sense, but
that is not the specific meaning of "shared memory."

In any event, the only way that I, personally, know of to
get the total memory used by *all* processes in the system
is from the output of "sar". Unfortunately, this will not
show you the per-process memory usage. But it will give you
a clue to to how much paging activity is going on, if any.
The OS typically uses an LRU algorithm (least recently
used), to page out parts of the data space for processes.
On the projects on which I work, we typically tolerate
up to 5 pgin/pgout per second, with peaks during times
when statistics are being gathered.

If you want finer granularity, do a "size" on the executable,
find out the "text, data and bss" sizes and break out your
calulator or spread*** :)

If one forked process starts writing to this Data-space, the
>> mem-page in question is copied (copy-on-write) and the process gets
>> its own private copy.
>> I think, that top does not tell me, how much mempages of the 24m are
>> actually shared with others or private space.

No, it does not ... see above.

>>
>
> Your "top" looks only a little like mine: different column headings, etc.
> But I would infer that these processes all share 8760K bytes and the
> rest is probably data and stack and not shared. Another possibility is
> that these are threads and not separate processes) and they share
> everything, and the 8760K is just system libraries shared with other
> processes not listed.
>
> 

-- 
"It is impossible to make anything foolproof
because fools are so ingenious"
  - A. Bloch