Re: RSS accounting (was: Re: 2.6.19-rc1-mm1)



On Tue, 2006-10-10 at 15:14 +0200, Peter Zijlstra wrote:

We need to consider at least if any of the following are part of rss:
* VM_IO io mmaped device stuff
* Non-linear mappings
* Shared hugetlb memory that shares pagetables
* Shared hugetlb memory
* Hugetlb memory in general
* Shared normal memory that shares pagetables
* Shared normal memory (file backed; eg pagecache)
* Shared normal memory (anonymous/non-file-backed)
* Sysv/ipc shared memory
* Not shared normal memory


So we have three cases where RSS matters besides presenting a number to
user-space;
- shared page tables
- containers
- rlimit

Preferably they will all share a common definition of what RSS is; since
containers must account shared pages somehow (not doing so would open up
a large hole to DoS the other containers) and the container case can be
argued to be an extension of the rlimit case we cannot just ignore them.

As then what to do with them, I've yet to figure out. Some random bit
floating in my brain;
- VM_IO can be discarted, its not actual memory

agreed (although I think we do count it today; this is half of what
makes X look so bloated, next to firefox ;)


- hugetlb is memory too, so I'd not special case this (other than the
different unit of accounting)

agreed again; personally I don't think hugetlb memory should be special;
especially with all the libhugetlbfs work on making it real easy for
apps to use; the more it's used the more people would notice something
funky with it.


- shared mapped pages could be accounted on vma level, since both
containers have access to the same file, there is already an imbalance,
so I'd not worry about the 1%-99% usage scenario here.

or one level below; if you count it in the actual PTE page then the
sharing case will "just work". It's a trick question; do you count it as
100% or do you count it as 100% / number of sharers.


- regular, non mapped, pagecache pages however have no owner - what to
do. (fake vma - which would result in each container paying equally for
all these pages?)

if they're clean I wouldn't count them to anything actually


Anyway, I'd rather not break RSS twice, once now because we don't quite
know what to do, and later when we do get an acceptable mm container and
have to include shared memory in one way or the other.


RSS of a container versus RSS of a process is an interesting question
for sure ;)


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



Relevant Pages

  • RSS accounting (was: Re: 2.6.19-rc1-mm1)
    ... RSS; 2.6.19-rc1-mm1 has a somewhat different one. ... remove more from RSS to help save memory and pagefaults (both help ... Shared hugetlb memory that shares pagetables ... containers must account shared pages somehow (not doing so would open up ...
    (Linux-Kernel)
  • Re: 2.6.19-rc1-mm1
    ... RSS; 2.6.19-rc1-mm1 has a somewhat different one. ... remove more from RSS to help save memory and pagefaults (both help ... Shared hugetlb memory that shares pagetables ... Shared normal memory ...
    (Linux-Kernel)
  • [RFC][PATCH][0/4] Memory controller (RSS Control) (
    ... This patch applies on top of Paul Menage's container patches posted at ... It implements a controller within the containers framework for limiting ... The memory controller was discussed at length in the RFC posted to lkml ...
    (Linux-Kernel)
  • Re: [patch00/05]: Containers(V2)- Introduction
    ... This job if run unconstrained could step over most of the memory present ... But when the same job is run inside containers then the backup ... We already have such a functionality in the kernel its called a cpuset. ... translate a single node machine into N different nodes. ...
    (Linux-Kernel)
  • Re: Creating STL Containers in Shared Memory
    ... > parse a data file and store data to my containers is increasing with each ... memory, and then ensuring that all of the containers use a custom memory ... allocator that manages that shared memory space. ... While mmap can take an address to map the memory, ...
    (comp.unix.solaris)