Re: VST and Sched Load Balance

From: Ingo Molnar (mingo_at_elte.hu)
Date: 04/07/05

  • Next message: Linus Torvalds: "Re: Kernel SCM saga.." 
    Date:	Thu, 7 Apr 2005 17:10:24 +0200
To: Srivatsa Vaddagiri <vatsa@in.ibm.com>

    * Srivatsa Vaddagiri <vatsa@in.ibm.com> wrote:

    > Hi,
    > VST patch (http://lwn.net/Articles/118693/) attempts to avoid useless
    > regular (local) timer ticks when a CPU is idle.
    >
    > I think a potential area which VST may need to address is scheduler
    > load balance. If idle CPUs stop taking local timer ticks for some
    > time, then during that period it could cause the various runqueues to
    > go out of balance, since the idle CPUs will no longer pull tasks from
    > non-idle CPUs.
    >
    > Do we care about this imbalance? Especially considering that most
    > implementations will let the idle CPUs sleep only for some max
    > duration (~900 ms in case of x86).

    yeah, we care about this imbalance, it would materially change the
    scheduling logic, which side-effect we dont want. Interaction with VST
    is not a big issue right now because this only matters on SMP boxes
    which is a rare (but not unprecedented) target for embedded platforms.

    One solution would be to add an exponential backoff would be ok (as Nick
    suggested too), not an unconditional 'we wont fire a timer interrupt for
    the next 10 seconds' logic. It still impacts scheduling though.

    Another, more effective, less intrusive but also more complex approach
    would be to make a distinction between 'totally idle' and 'partially
    idle or busy' system states. When all CPUs are idle then all timer irqs
    may be stopped and full VST logic applies. When at least one CPU is
    busy, all the other CPUs may still be put to sleep completely and
    immediately, but the busy CPU(s) have to take over a 'watchdog' role,
    and need to run the 'do the idle CPUs need new tasks' balancing
    functions. I.e. the scheduling function of other CPUs is migrated to
    busy CPUs. If there are no busy CPUs then there's no work, so this ought
    to be simple on the VST side. This needs some reorganization on the
    scheduler side but ought to be doable as well.

            Ingo
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Linus Torvalds: "Re: Kernel SCM saga.."

    Relevant Pages

    • Re: [PATCH] i386 No-Idle-Hz aka Dynamic-Ticks 3
      ... > virtualization benefit if idle CPUs can cut off useless timer ticks. ... > seems to me that VST has already addressed all these to a big extent. ...
      (Linux-Kernel)
    • Re: CONFIG_NO_HZ breaks blktrace timestamps
      ... git remote --add linus ... Sometimes cpu_idle_wait gets stuck because it might miss CPUS that are ... This is common on bootup when switching cpu idle governors. ... Switch global idle handler to always present default_idle ...
      (Linux-Kernel)
    • Re: CONFIG_IRQBALANCE for AMD64?
      ... irqbalance approximates irq processing load by irq count, ... Utilize idle CPUs where possible to shoulder the load. ... > all the time in busy scenarios (and non-busy scenarios are boring wrt irq ...
      (Linux-Kernel)
    • Re: Soft lockup regression from todays sched.git merge.
      ... I'm getting will have 256 cpus. ... kind of global lock you complain about above, ... Every time irq_enteris done on an idle CPU we do: ... That's the price we pay for global jiffies and shoddy ...
      (Linux-Kernel)
    • Re: [GIT PULL] cputime patch for 2.6.30-rc6
      ... more idle time than elapsed time. ... compare elapsed time * #cpus with the idle time. ... points where the number of cpus did not change. ... uptime and idle) should be amenable to simple interpretation. ...
      (Linux-Kernel)