Re: RCU scaling on large systems

From: Jesse Barnes (jbarnes_at_engr.sgi.com)
Date: 05/03/04

  • Next message: Stephen Smalley: "[PATCH][SELINUX] Re-open descriptors closed on exec by SELinux to /dev/null" 
    To: linux-kernel@vger.kernel.org, paulmck@us.ibm.com
Date:	Mon, 3 May 2004 09:39:11 -0700

    On Sunday, May 2, 2004 11:28 am, Paul E. McKenney wrote:
    > From your numbers below, I would guess that if you have at least
    > 8 CPUs per NUMA node, a two-level tree would suffice. If you have
    > only 4 CPUs per NUMA node, you might well need a per-node level,
    > a per-4-nodes level, and a global level to get the global lock
    > contention reduced sufficiently.

    Actually, only 2, but it sounds like your approach would work.

    > Cute! However, it is not clear to me that this approach is
    > compatible with real-time use of RCU, since it results in CPUs
    > processing their callbacks less frequently, and thus getting
    > more of them to process at a time.

    I think it was just a proof-of-concept--the current RCU design obviously
    wasn't designed with this machine in mind :).

    > But it is not clear to me that anyone is looking for realtime
    > response from a 512-CPU machine (yow!!!), so perhaps this
    > is not a problem...

    There are folks that would like realtime (or close to realtime) response on
    such systems, so it would be best not to do anything that would explicitly
    prevent it.

    > This patch certainly seems simple enough, and I would guess that
    > "jiffies" is referenced often enough that it is warm in the cache
    > despite being frequently updated.
    >
    > Other thoughts?

    On a big system like this though, won't reading jiffies frequently be another
    source of contention?

    Jesse
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Stephen Smalley: "[PATCH][SELINUX] Re-open descriptors closed on exec by SELinux to /dev/null"

    Relevant Pages

    • Attempted summary of "RT patch acceptance" thread, take 2
      ... Common wisdom dictates that realtime operating systems, ... there are applications that can be satisfied ... only by custom hardware implementations. ... If so, how many CPUs? ...
      (Linux-Kernel)
    • Re: scheduler scalability - cgroups, cpusets and load-balancing
      ... The 'sched_load_balance' flag is an attribute of cpusets. ... the kernel does not do normal load balancing on select CPUs, ... as those CPUs dedicated to realtime use. ...
      (Linux-Kernel)
    • Re: [RFC&PATCH] Alternative RCU implementation
      ... >> IPIs enabled on the realtime CPU?). ... We may have several cpus running ... about RCU is the right way to go IMO. ... Anything that poll other cpus and has read-side overheads will likely ...
      (Linux-Kernel)
    • Re: [CPUISOL] CPU isolation extensions
      ... Ingo - see question at end of message. ... I'd like to extend it further to avoid kernel activity on those CPUs as much as possible. ... It looks like you have three additional tweaks for realtime in this ...
      (Linux-Kernel)
    • Re: [PATCH] 2.6 workaround for Athlon/Opteron prefetch errata
      ... This buys nothing on CPUs which don't have the ... We have always built kernels that contained the support for older chips. ... difference in the real world between PIV and PIV with 100 bytes of extra ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)