Re: [Ksummit-2008-discuss] Delayed interrupt work, thread pools

On Wed, 2008-07-02 at 13:02 +0200, Andi Kleen wrote:
Benjamin Herrenschmidt <benh@xxxxxxxxxxxxxxxxxxx> writes:

how much of this would be obsoleted if we had irqthreads ?

I'm not sure irqthreads is what I want...

First, can they call handle_mm_fault ? (ie, I'm not sure precisely what
kind of context those operate into).

Interrupt threads would be kernel threads and kernel threads
run with lazy (= random) mm and calling handle_mm_fault on that
wouldn't be very useful because you would affect a random mm.

That isn't a big issue. handle_mm_fault() takes the mm as an argument
(like when called from get_user_pages()) and if there's anything fishy I
can always attach/detach the mm to the thread. Been done before, works
fine.

Ok you could force them to run with a specific MM, but that would
cause first live time issues with the original MM (how could you
ever free it?) and also increase the interrupt handling latency
because the interrupt would be a nearly full blown VM context
switch then.

handle_mm_fault() shouldn't need an mm context switch. I can just
refcount while I have a ref. to the mm in my queue. I can deal with
lifetime, that isn't a big issue.

I also think interrupts threads are a bad idea in many cases because
their whole "advantage" over classical interrupts is that they can
block. Now blocking can be usually take a unbounded potentially long
time.

Yes, that's what I explain in the rest of my mail. That plus the fact
that I need to context switch the SPU to other contexts while we block.

.../...

I agree with most of your points, which is why I believe interrupt
threads aren't a good option for me.

Interrupts for "normal" events will be handled in a short/bounded time.

Interrupts coming from SPU page faults will be deferred to a thread from
a pool (which can need more time if none is available, ie work queue ->
allocate more, or just wait on one to free up, the stategy here is to be
defined).

It's not a problem to have them delayed. I can context switch a faulting
SPU to some other task and switch it back later when the fault is
serviced. Anything time critical shouldn't operate on fault-able memory
in the first place :-)

So I need at most one kernel thread per SPU context for handling the
faults. The idea of the thread pools is that most of the time, I don't
take faults, and thus I don't nearly need as many threads in practice.
Thus having a pool that can dynamically grow or shrink based on pressure
would make sense.

Ben.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: LinuxPPS & spinlocks
    ... I meant that if you have to lock between user context and interrupt ... Ok, I see, but how you can get your PPS source data struct starting ...
    (Linux-Kernel)
  • Re: GRU driver feedback
    ... "atomic pte lookup" which looks up the pte atomically using similar ... Existence of the mm is guaranteed thru an indirect path. ... struct cannot go away until the GRU context that caused the interrupt ... When the GRU hardware sends an interrupt, ...
    (Linux-Kernel)
  • Re: [KVM-RFC PATCH 1/2] eventfd: add an explicit srcu based notifier interface
    ... interrupt from eth to host wakes the cpu out of idle, ... softirq-net-rx in interrupt context to NAPI-poll the ethernet ... The interface is just ugly IMO. ... that must support being invoked from atomic-context. ...
    (Linux-Kernel)
  • Re: Concurrent Sequential
    ... Our "trick" lets you remove a whack of context switch overhead. ... instructions to perform the interrupt service. ... "Context" does not need to be stored on the stack, ...
    (comp.arch)
  • Re: LinuxPPS & spinlocks
    ... does nothing (if we are already in an interrupt handler). ... context, but doing like this I will disable interrupts ... protect syscalls from each others and from pps_register/unregister, ... Note that *data* is protected by a lock, ...
    (Linux-Kernel)