Re: [RFC v1][PATCH]page_fault retry with NOPAGE_RETRY

On Thu, Nov 27, 2008 at 11:00:07AM +0100, Peter Zijlstra wrote:
On Thu, 2008-11-27 at 01:28 -0800, Mike Waychison wrote:

Correct. I don't recall the numbers from the pathelogical cases we were
seeing, but iirc, it was on the order of 10s of seconds, likely
exascerbated by slower than usual disks. I've been digging through my
inbox to find numbers without much success -- we've been using a variant
of this patch since 2.6.11.

We generally try to avoid such things, but sometimes it a) can't be
easily avoided (third party libraries for instance) and b) when it hits
us, it affects the overall health of the machine/cluster (the monitoring
daemons get blocked, which isn't very healthy).

If its only monitoring, there might be another solution. If you can keep
the required data in a separate (approximate) copy so that you don't
need mmap_sem at all to show them.

If your mmap_sem is so contended your latencies are unacceptable, adding
more users to it - even statistics gathering, just isn't going to cure
the situation.

Furthermore, /proc code usually isn't written with performance in mind,
so its usually simple and robust code. Adding it to a 'hot'-path like
you're doing doesn't seem advisable.

Also, releasing and re-acquiring mmap_sem can significantly add to the
cacheline bouncing that thing already has.

Yes, it would be nice to reduce mmap_sem load regardless of any other
fixes or problems. I guess they're not very worried about cacheline
bouncing but more about hold time (how many sockets in these systems?
4 at most?)

I guess it is the pagemap stuff that they use most heavily?

pagemap_read looks like it can use get_user_pages_fast. The smaps and
clear_refs stuff might have been nicer if they could work on ranges
like pagemap. Then they could avoid mmap_sem as well (although maps
would need to be sampled and take mmap_sem I guess).

One problem with dropping mmap_sem is that it hurts priority/fairness.
And it opens a bit of a (maybe theoretical but not something to completely
ignore) forward progress hole AFAIKS. If mmap_sem is very heavily
contended, then the refault is going to take a while to get through,
and then the page might get reclaimed etc).



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: eBay item removed *after* payment...
    ... Reply from Minister of Health ... The situation regarding blood glucose testing strips has arisen ... taken this to mean that home blood glucose monitoring is not ...
    (uk.people.consumers.ebay)
  • Bushco hates 9/11 Responders
    ... Loch Ness monster in terms of being able to find this monitoring," said Jon ... Programs were developed to check on the health of every other group that ... Officials involved told The News the feds barred their workers from ... An HHS spokesman insisted the program had not been lost, ...
    (alt.politics.bush)
  • Re: [RFC v1][PATCH]page_fault retry with NOPAGE_RETRY
    ... I've been digging through my ... inbox to find numbers without much success -- we've been using a variant ... it affects the overall health of the machine/cluster (the monitoring ... If its only monitoring, ...
    (Linux-Kernel)
  • RE: Attribute seek error rate
    ... You probably have a program running on your machine that is monitoring your ... disk drive's health. ...
    (microsoft.public.windowsxp.perform_maintain)