Re: /proc reliability & performance

From: Albert Cahalan (albert_at_users.sf.net)
Date: 10/17/03

  • Next message: William Lee Irwin III: "Re: /proc reliability & performance" 
    To: Brian McGroarty <brian@mcgroarty.net>
Date:	17 Oct 2003 00:31:14 -0400

    On Thu, 2003-10-16 at 23:24, Brian McGroarty wrote:
    > On Thu, Oct 16, 2003 at 10:07:18PM -0400, Albert Cahalan wrote:
    > > I created a process with 360 thousand threads,
    > > went into the /proc/*/task directory, and did
    > > a simple /bin/ls. It took over 9 minutes on a
    > > nice fast Opteron. (it's the same at top-level
    > > with processes, but I wasn't about to mess up
    > > my system that much)
    >
    > Are there many cases where the /proc directory
    > contents are read in this fashion?

    Sure. Run any of: top, ps, lsof, fuser...

    > I'd be more curious about how performance fares
    > with reading a thousand entries by name with 1k
    > processes and with 360k processes.

    Judging by the crazy example and the observation
    that an O(n*n) algorithm is involved, directory
    reads on that very fast machine should get annoying
    once you have a few thousand processes. They'd be
    perceptable one-by-one, which adds up when you have
    multiple reads due to scripts, top, or multiple
    users.

    Anyway, it's not just about performance! That's
    only half of the problem. The other half is
    reliability. The way /proc works is this:

    Count tasks as you read them. The number is
    your directory offset. Return a few dozen entries
    at a time. For each read, you'll need to find
    back your place. You do this by counting tasks
    until you reach your offset. Of course, tasks
    will have been created and destroyed between
    reads, so who knows where you'll continue from?

    That's simply not reliable.

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: William Lee Irwin III: "Re: /proc reliability & performance"

    Relevant Pages

    • Re: [PATCH] (18/22) task_thread_info - part 2/4
      ... > Nothing subtle with it, especially since this is the only place with any ... we could have multiple versions setup_thread_infoand every one ... small (50KB in 7 patches). ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: [Announce] Intel PRO/Wireless 2100 802.11b driver
      ... Multiple interfaces should be supported with the current code; ... potential ipw2100 devices than on a per device basis. ... interested in helping us improve the driver. ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: [discuss] Re: kgdb for x86_64 2.6 kernels
      ... > It doesn't contain any assert stuff. ... I have split it into multiple parts to ... When do you have a heart between your knees? ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: [PATCH] iSeries virtual disk
      ... These functions do not look guarded against multiple openers. ... referencing the struct scatterlist elements. ... instead of duplicating the same cleanup code multiple times ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: Database design pattern question
      ... Some of the data comes from different sources, with different levels of reliability. ... The disadvantage of this approach would be that I could no longer simply merge multiple copies of the database by forming unions of the event tables. ... When it does it will be necessary to replace the disk drive and then rebuild your database using your most recent backup and a transaction file containing the transactions that have been processed since the backup was made. ...
      (comp.databases)