Re: 2.6.12 Performance problems

From: Danial Thom (danial_thom_at_yahoo.com)
Date: 08/23/05

  • Next message: James Bottomley: "Re: [PATCH 2.6.12.5 1/2] lib: allow idr to be used in irq context" 
    Date:	Tue, 23 Aug 2005 10:10:28 -0700 (PDT)
To: Helge Hafting <helge.hafting@aitel.hist.no>

    --- Helge Hafting <helge.hafting@aitel.hist.no>
    wrote:

    > Danial Thom wrote:
    >
    > >--- Jesper Juhl <jesper.juhl@gmail.com> wrote:
    > >
    > >
    > >
    > >>On 8/21/05, Danial Thom
    > <danial_thom@yahoo.com>
    > >>wrote:
    > >>
    > >>
    > >>>I just started fiddling with 2.6.12, and
    > >>>
    > >>>
    > >>there
    > >>
    > >>
    > >>>seems to be a big drop-off in performance
    > >>>
    > >>>
    > >>from
    > >>
    > >>
    > >>>2.4.x in terms of networking on a
    > >>>
    > >>>
    > >>uniprocessor
    > >>
    > >>
    > >>>system. Just bridging packets through the
    > >>>machine, 2.6.12 starts dropping packets at
    > >>>~100Kpps, whereas 2.4.x doesn't start
    > >>>
    > >>>
    > >>dropping
    > >>
    > >>
    > >>>until over 350Kpps on the same hardware
    > >>>
    > >>>
    > >>(2.0Ghz
    > >>
    > >>
    > >>>Opteron with e1000 driver). This is pitiful
    > >>>prformance for this hardware. I've
    > >>>increased the rx ring in the e1000 driver to
    > >>>
    > >>>
    > >>512
    > >>
    > >>
    > >>>with little change (interrupt moderation is
    > >>>
    > >>>
    > >>set
    > >>
    > >>
    > >>>to 8000 Ints/second). Has "tuning" for MP
    > >>>destroyed UP performance altogether, or is
    > >>>
    > >>>
    > >>there
    > >>
    > >>
    > >>>some tuning parameter that could make a
    > >>>
    > >>>
    > >>4-fold
    > >>
    > >>
    > >>>difference? All debugging is off and there
    > >>>
    > >>>
    > >>are
    > >>
    > >>
    > >>>no messages on the console or in the error
    > >>>
    > >>>
    > >>logs.
    > >>
    > >>
    > >>>The kernel is the standard kernel.org
    > dowload
    > >>>config with SMP turned off and the intel
    > >>>
    > >>>
    > >>ethernet
    > >>
    > >>
    > >>>card drivers as modules without any other
    > >>>changes, which is exactly the config for my
    > >>>
    > >>>
    > >>2.4
    > >>
    > >>
    > >>>kernels.
    > >>>
    > >>>
    > >>>
    > >>If you have preemtion enabled you could
    > disable
    > >>it. Low latency comes
    > >>at the cost of decreased throughput - can't
    > >>have both. Also try using
    > >>a HZ of 100 if you are currently using 1000,
    > >>that should also improve
    > >>throughput a little at the cost of slightly
    > >>higher latencies.
    > >>
    > >>I doubt that it'll do any huge difference,
    > but
    > >>if it does, then that
    > >>would probably be valuable info.
    > >>
    > >>
    > >>
    > >Ok, well you'll have to explain this one:
    > >
    > >"Low latency comes at the cost of decreased
    > >throughput - can't have both"
    > >
    > >
    > Configuring "preempt" gives lower latency,
    > because then
    > almost anything can be interrupted (preempted).
    > You can then
    > get very quick responses to some things, i.e.
    > interrupts and such.

    I think part of the problem is the continued
    misuse of the word "latency". Latency, in
    language terms, means "unexplained delay". Its
    wrong here because for one, its explainable. But
    it also depends on your perspective. The
    "latency" is increased for kernel tasks, while it
    may be reduced for something that is getting the
    benefit of preempting the kernel. So you really
    can't say "the price of reduced latency is lower
    throughput", because thats simply backwards.
    You've increased the kernel tasks latency by
    allowing it to be pre-empted. Reduced latency
    implies higher efficiency. All you've done here
    is shift the latency from one task to another, so
    there is no reduction overall, in fact there is
    probably a marginal increase due to the overhead
    of pre-emption vs doing nothing.

    DT

                    
    ____________________________________________________
    Start your day with Yahoo! - make it your home page
    http://www.yahoo.com/r/hs
     
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: James Bottomley: "Re: [PATCH 2.6.12.5 1/2] lib: allow idr to be used in irq context"

    Relevant Pages

    • Re: [PATCH] Avoid moving tasks when a schedule can be made.
      ... > not interrupted for more than Xus, including scheduling latency. ... current x86 systems running the -rt kernel with pure IRQ load. ... > if you "break" that latency by splitting it into two interrupts, ...
      (Linux-Kernel)
    • Re: Interrupt statistics?
      ... dominated by the clk and rtc interrupts as shown below. ... How much latency do the ... an idle system, this should be fine although it does add to the latency ...
      (freebsd-current)
    • Re: Interrupt statistics?
      ... I am very interested in our idle load characteristics. ... most systems I've analyzed have an average idle interrupt rate of about ... Since these are both clocks, I assume the arrival of their interrupts are ... states which can have as high a latency as a few hundred microseconds. ...
      (freebsd-current)
    • Re: IRQF_DISABLED problem
      ... interrupts. ... the most expensive thing in terms of adding latency to the system. ... I don't know if Fedora still does this. ...
      (Linux-Kernel)
    • Re: polling(4) rocks!
      ... > added latency typically ends up being around 1ms. ... Latency is affected if in the case of using interrupts, ... than for polling. ... If you plot a latency versus pps graph, ...
      (freebsd-net)