Re: memory resource accounting (was Re: [RFC, PATCH 0/5] Going forward with Resource Management - A cpu controller)

Nick,

On Wed, Aug 09, 2006 at 12:19:41AM +1000, Nick Piggin wrote:
[snip]

What's the sucking semantics on exit? I haven't looked much at the
existing memory controllers going around, but the implementation I
imagine looks something like this (I think it is conceptually similar
to the basic beancounters idea):

What you suggests implies an over-simplification of how memory is used in the
system, and doesn't take into account sharing and caching.

Namely:

- anyone who allocates a page for anything gets charged for that page.
Except interrupt/softirq context. Could we ignore these for the moment?

This does give you kernel (slab, pagetable, etc) allocations as well as
userspace. I don't like the idea of doing controllers for inode cache
and controllers for dentry cache, etc, etc, ad infinitum.

So, are you suggesting that the user (or container) that initially looked up
some directory /var/dir, will stay billed for this memory until
- all users of all subdirectories /var/dir/a/b/c/d/e/f/g/h/i etc.
are gone, and
- dentry cache has been shrunk because of memory pressure?

It is unfair.
But more than that:
one of the goals of resource accounting and restrictions is to give users the
idea of how much resources they are using. Then, when they start to use more
than their allotment, they should be given the opportunity to consider what
they are doing and reduce resource usage.

In my opinion, to make resource accounting useful, serious efforts should
be made not to bill anyone for resources which he isn't really using and has
no control over releasing them.


- each struct page has a backpointer to its billed container. At the mm
summit Linus said he didn't want back pointers, but I clarified with him
and he isn't against them if they are easily configured out when not using
memory controllers.

The same thing: if one user mapped and then released some pages from a shared
library, should he be billed for these pages until all other users unmapped
this library, and page cache has been shrunk?

Best regards
Andrey
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: [ProbableSpam] Re: [RFC, PATCH 0/5] Going forward with Resource Management - A cpu controll
    ... Should task-group be changeable after set/inherited once? ... e.g. shared memory or used kernel memory is hard to recalculate. ... If userspace wants an application's resource usage to depend on the kind ... But lets not worry about this aspect right now until the controllers are sorted ...
    (Linux-Kernel)
  • Re: [PATCH] Add a document describing the resource counter abstraction (v2)
    ... of arbitrary resource (and it already does this for memory controller). ... However, it is about to be used in other resources controllers (swap, ... Basic accounting routines ...
    (Linux-Kernel)
  • Re: Database performance
    ... Memory: Depending on your processing needs, if a small search set can ... you can and should by all means run ext2 on your scratch ... Different RAID levels are appropriate to different tasks. ... Multiple controllers help. ...
    (Debian-User)
  • [RFC] Revised CKRM release
    ... The Class-based Resource Management project is happy to release the ... A filesystem-based user interface, proposed by Rik van Riel, to ... useful for controlling groups of sockets. ... Resource controllers are now explicitly associated with a ...
    (Linux-Kernel)
  • Re: C++ sucks for games
    ... Just a typo that can be horrendous to ... but if your memory management is scattered and out of control, ... >>implement raii, and i think that's a serious flaw. ... resource - and resource can be anything at all. ...
    (comp.lang.lisp)