Re: [RFC][PATCH] Reduce ext3 allocate-with-reservation lock latencies

From: Mingming Cao (cmm_at_us.ibm.com)
Date: 04/29/05

  • Next message: Mingming Cao: "[PATCH] Reduce ext3 allocate-with-reservation lock latencies" 
    To: Lee Revell <rlrevell@joe-job.com>
Date:	Thu, 28 Apr 2005 23:18:10 -0700

    On Thu, 2005-04-28 at 11:34 -0700, Mingming Cao wrote:
    > On Thu, 2005-04-28 at 12:12 -0400, Lee Revell wrote:
    > > On Thu, 2005-04-28 at 00:37 -0700, Mingming Cao wrote:
    > > > On Wed, 2005-04-27 at 23:45 -0400, Lee Revell wrote:
    > > > > On Fri, 2005-04-22 at 15:10 -0700, Mingming Cao wrote:
    > > > > > Please review. I have tested on fsx on SMP box. Lee, if you got time,
    > > > > > please try this patch.
    > > > >
    > > > > I have tested and this does fix the problem. I ran my tests and no ext3
    > > > > code paths showed up on the latency tracer at all, it never went above
    > > > > 33 usecs.
    > > > >
    > > > Thanks, Lee.
    > > >
    > > > The patch survived on many fsx test over 20 hours on a 2cpu machine.
    > > > Tested the patch on the same machine with tiobench (1-64 threads), and
    > > > untar a kernel tree test, no regression there.
    > > > However I see about 5-7% throughput drop on dbench with 16 threads. It
    > > > probably due to the cpu cost that we have discussed.
    > >
    > > Hmm, I guess someone needs to test it on a bigger system. AFAICT this
    > > should improve SMP scalability quite a bit. Maybe that lock is rarely
    > > contended.
    >
    > Probably. I will try it on a relatively full filesystem(where bitmap
    > scan to find a free block takes time), and on a 4 cpu box with many
    > threads allocating at the same time.

    I run 64 threads dbench on a 4way box on a 90% full filesystem, patched
    kernel is slightly faster. I think part of the reason is on a 90% full
    filesystem, block bitmap scan will take a long time. Thus drop the lock
    before bitmap scan will help, although there are some overhead of
    drop/re-grab lock and extra window search/link/unlink before we found a
    window with free block. On a freshly created filesystem where the
    regression is seen, the bitmap scan takes shorter time, thus the
    overhead of drop/re-grab lock and extra window search/link/unlink
    probably is probably be outstanding.

    I also run 1000 dd tests in parallel, each of them writing to different
    file(2M size). Seems no gain or no regression.

    Anyway, we should not hold the global reservation lock while scanning
    the bitmap, that's a scalability issue and a latency issue. To drop the
    lock during bitmap scan we need to do extra work, I don't think it's
    avoidable. Andrew, could you add it to mm tree? I will re-send the
    patch with the change log.

    Thanks,
    Mingming

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Mingming Cao: "[PATCH] Reduce ext3 allocate-with-reservation lock latencies"

    Relevant Pages

    • Re: 2.6.17-rc6-mm1 -- BUG: possible circular locking deadlock detected!
      ... What happened is that the rwlock_init() 'lock type keys' ... for the two runlists were 'merged': ... lcnbmp_lock locks accesses to the cluster bitmap (i.e. the physical blocks ... The locks are taken when a cluster is being allocated / an mft record is ...
      (Linux-Kernel)
    • Re: Having trouble with multiple threads and a shared resource
      ... I read that you should use a static private object as a lock, ... access between the threads to the bitmap. ... private void _replayPathTaken_Click ... HighlightSelectedCells method need access to MyBitmap and sometimes they ...
      (microsoft.public.dotnet.languages.csharp)
    • Flickering
      ... I figure that if I lock the bitmap that I can draw on it without the painter ... FormGraphics = this.CreateGraphics; ... protected override void OnPaintBackground ...
      (microsoft.public.dotnet.languages.csharp)
    • Re: SetPixel() / GetPixel() for Format64bppArgb in VB using LockBits
      ... I didn't know that one can lock just some parts of the bitmap. ... So you mean if one has to set 1 pixel, ... Find great Windows Forms articles in Windows Forms Tips and Tricks ...
      (microsoft.public.dotnet.framework.drawing)
    • Re: Can you lock a slide in PowerPoint?
      ... > to group all the contents togetherbut this doesnt really lock them. ... You can export the slide to a bitmap then import the bitmap into a new ... Steve Rindsberg, PPT MVP ...
      (microsoft.public.powerpoint)
    Loading