Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

Avi Kivity <avi@xxxxxxxxxx> writes:

On 03/15/2010 10:07 AM, Balbir Singh wrote:

Yes, it is a virtio call away, but is the cost of paying twice in
terms of memory acceptable?

Usually, it isn't, which is why I recommend cache=off.

Hi Avi. One observation about your recommendation for cache=none:

We run hosts of VMs accessing drives backed by logical volumes carved out
from md RAID1. Each host has 32GB RAM and eight cores, divided between (say)
twenty virtual machines, which pretty much fill the available memory on the
host. Our qemu-kvm is new enough that IDE and SCSI drives with writeback
caching turned on get advertised to the guest as having a write-cache, and
FLUSH gets translated to fsync() by qemu. (Consequently cache=writeback
isn't acting as cache=neverflush like it would have done a year ago. I know
that comparing performance for cache=none against that unsafe behaviour
would be somewhat unfair!)

Wasteful duplication of page cache between guest and host notwithstanding,
turning on cache=writeback is a spectacular performance win for our guests.
For example, even IDE with cache=writeback easily beats virtio with
cache=none in most of the guest filesystem performance tests I've tried. The
anecdotal feedback from clients is also very strongly in favour of
cache=writeback.

With a host full of cache=none guests, IO contention between guests is
hugely problematic with non-stop seek from the disks to service tiny
O_DIRECT writes (especially without virtio), many of which needn't have been
synchronous if only there had been some way for the guest OS to tell qemu
that. Running with cache=writeback seems to reduce the frequency of disk
flush per guest to a much more manageable level, and to allow the host's
elevator to optimise writing out across the guests in between these flushes.

Cheers,

Chris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • [PATCH 2/7] lguest: documentation pt II: Guest
    ... Documentation: The Guest ... * ring buffer of stored hypercalls which the Host will run though next time we ... * The simplest way of implementing these would be to have "turn interrupts ... static void restore_fl ...
    (Linux-Kernel)
  • [PATCH 2/7] lguest documentation:Chapter II
    ... Documentation: The Guest ... * ring buffer of stored hypercalls which the Host will run though next time we ... * The simplest way of implementing these would be to have "turn interrupts ... static void restore_fl ...
    (Linux-Kernel)
  • [patch 1/6] Guest page hinting: core + volatile page cache.
    ... The guest page hinting patchset introduces code that passes guest ... page usage information to the host system that virtualizes the ... Volatile: The page content is useful to the guest but not ... The host can discard the page but has to deliver a ...
    (Linux-Kernel)
  • [patch 1/6] Guest page hinting: core + volatile page cache.
    ... The guest page hinting patchset introduces code that passes guest ... page usage information to the host system that virtualizes the ... Volatile: The page content is useful to the guest but not ... The host can discard the page but has to deliver a ...
    (Linux-Kernel)
  • [patch 1/6] Guest page hinting: core + volatile page cache.
    ... The guest page hinting patchset introduces code that passes guest ... page usage information to the host system that virtualizes the ... Volatile: The page content is useful to the guest but not ... The host can discard the page but has to deliver a ...
    (Linux-Kernel)