/proc reliability & performance

From: Albert Cahalan (albert_at_users.sf.net)
Date: 10/17/03

  • Next message: Larry McVoy: "Re: /proc reliability & performance" 
    To: linux-kernel mailing list <linux-kernel@vger.kernel.org>
Date:	16 Oct 2003 22:07:18 -0400

    I created a process with 360 thousand threads,
    went into the /proc/*/task directory, and did
    a simple /bin/ls. It took over 9 minutes on a
    nice fast Opteron. (it's the same at top-level
    with processes, but I wasn't about to mess up
    my system that much)

    OK, that's a bit extreme, but it does show a
    scalability problem. There is an O(n*n)
    algorithm in there. Here is a proposed fix:

    Tie directory readers to a task_struct (or to
    some of the PID tracking structs), so that
    a directory reader is on a list. When a task
    exits, move the list of directory readers on
    to a neighboring task.

    That is O(1) on task exit, and generally O(n)
    for the whole /proc or /proc/42/task read.
    It's O(1) per step of the read, excepting
    where multiple directory readers wind up at
    the same location.

    Another benefit is that it is reliable as
    long as tasks don't move around on the lists.
    Each task will appear at most once, and will
    appear exactly once if it doesn't start or
    exit during the directory scan.

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Larry McVoy: "Re: /proc reliability & performance"

    Relevant Pages

    • Re: /proc reliability & performance
      ... > Tie directory readers to a task_struct (or to ... > That is Oon task exit, ... > long as tasks don't move around on the lists. ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: [opensuse] postfix master.cf question
      ... I tried with a custom script but i cant find how to get the mail to a variable to pass it to the mailfax command, i get the rest of the parameters, but no idea of how to get the mail itself ... The documentation also lists the variables that can be used in mailbox_command. ... echo Cannot save mail to file; exit $EX_TEMPFAIL;} ...
      (SuSE)
    • Re: A better way to split up a list
      ... It looks like there may be some problems with your code, such that it doesn't "work fine"...particularly in lists with 1, 2, or 3 elements. ... The below "chopupmoves2" function does what you describe, ... def chopupmoves2: ... while exit < len: ...
      (comp.lang.python)
    • A better way to split up a list
      ... Is there a better way to do the docstring task? ... '''creates a list of 3 lists from one list, ... with any remainder getting added to the ... while exit < len: ...
      (comp.lang.python)
    • Re: A better way to split up a list
      ... This is the first working version, and can proabably be "compacted" a ... def chopupmoves: ... '''creates a list of 3 lists from one list, ... while exit < len: ...
      (comp.lang.python)