Re: Analogue of Windows I/O completion ports on Linux?

From: David Schwartz (davids_at_webmaster.com)
Date: 09/27/05 

Date: Mon, 26 Sep 2005 21:33:15 -0700

"Peter T. Breuer" <ptb@oboe.it.uc3m.es> wrote in message
news:5a6m03-ivv.ln1@news.it.uc3m.es...

>> There's really no good way to lock in pages that haven't faulted in
>> yet.

> Eh? That's not at issue! The idea is to lock in those pages that HAVE
> faulted in!

    How does that help if the page you need to lock in hasn't faulted in
yet?

> Oh .. maybe you phrased that a little off. I think you mean
> that there's no good way to eagerly fault in all pages that you will
> ever want at startup, in order that they be locked in ram and thus
> avaialble for when you need them.

    Right. You can't avoid page faults during execution, so you have to
minimize the problems they cause.

>> It depends upon a lot of complicated factors included how much right your
>> application has to hog resources. Obviously, if you're the only important
>> application on the system, go right ahead.

> He thinks he is.

> I have some daemons locked in ram that need to do a tight loop
> servicing calls to and from the kernel. Swapping them out would be
> nasty, but the inner loop is all hat they really need.

    Until something unusual happens that requires code other than the inner
loop.

>> However, I have hear the complaint that systems work perfectly fine
>> on
>> test harnesses and handle huge load. But then when they're tested in the
>> real world, they get bursty and sometimes fall victim to lag from which
>> they
>> cannot catch up.

>> The problem is that the test code is repeating the same bits of
>> operations, so they stay hot in the processor caches and in memory. Then
>> real world clients hit infrequently used code that has to be faulted in
>> or

> Well, one can't force stuff into memory when it's not known what it
> will be! Goedel's theorem, and all that. All you can do is keep some
> memory available so that nothing else has to be swapped out when the
> time comes. How you do that is definitely tricky .. I agree.

    Right, that's why I don't generally recommend a single thread in a
'select' loop as a design model for servers. There are, of course, some
cases where it's the best approach. But that's not common.

>> I do not recommend this design. (Non-blocking I/O with 'select' in a
>> single thread.)

> I imagine the daemons exercise a sumple service cycle. It should all be
> there when it's needed.

    Sooner or later you'll need to do something that can fault that's not
easily avoidable. Consider reading a file.

    DS

Relevant Pages

  • Re: Analogue of Windows I/O completion ports on Linux?
    ... > that there's no good way to eagerly fault in all pages that you will ... > ever want at startup, in order that they be locked in ram and thus ... but the inner loop is all hat they really need. ... so they stay hot in the processor caches and in memory. ...
    (comp.os.linux.development.apps)
  • Re: Fast string operations
    ... > is why people use unsafe code now and then to use pointer arithmetic to ... > loop over arrays without all the unnecessary bounds checking. ... don't return the trimmed string". ... The customer perceives this as a memory leak. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Performance Improvement of complex data structure (hash of hashes of hashes)
    ... Anno Siegel wrote: ... >> Here is the code that I'm using to build up this data structure. ... loop through and increment the ... I know that memory allocation in C is expensive, ...
    (comp.lang.perl.misc)
  • Re: Cost of calling a standard library function
    ... It accesses/reads memory using esi 4 ... > safly move it within the cache, without having to go via ebx. ... try it the same thing on a different earlier CPU, ... should check it out...for "tight inner loop" stuff, ...
    (alt.lang.asm)
  • Re: Restoring a NorthStar Horizon, problems with SRAM board
    ... A-D FDC is a memory mapped IO device so I would expect some behavior ... loop read/write functions - which puts the system into a two instruction ... SRAM on the Vector Graphics Flashwriter video card located at $F000 ...
    (comp.sys.northstar)