Re: clearing filesystem cache for I/O benchmarks

From: Andrew Morton (akpm_at_osdl.org)
Date: 07/27/04

  • Next message: Denis Vlasenko: "Re: [PATCH 2.6.8-rc2] intel8x0.c to include CK804 audio support" 
    Date:	Mon, 26 Jul 2004 23:40:05 -0700
To: Benjamin Rutt <rutt.4+news@osu.edu>

    (Please don't remove people from the email recipient list when doing kernel
    work.)

    Benjamin Rutt <rutt.4+news@osu.edu> wrote:
    >
    > Andrew Morton <akpm@osdl.org> writes:
    >
    > > Benjamin Rutt <rutt.4+news@osu.edu> wrote:
    > >>
    > >> How can I purge all of the kernel's filesystem caches, so I can trust
    > >> that my I/O (read) requests I'm trying to benchmark bypass the kernel
    > >> filesystem cache?
    > >
    > > Either delete the benchmark test files or
    >
    > I'm not sure I follow. If I delete the benchmark files, I'll only
    > need to create them again later in order to do a read test, and I'll
    > have the same problem then, of how to eliminate the just-written-data
    > from cache.

    OK.

    > Thanks for the reference, I wasn't aware of that one. We are running
    > some 2.4 kernels in our storage cluster unfortunately so that won't be
    > usable for us everywhere. I take it POSIX_FADV_DONTNEED is ignored
    > under 2.4.

    posix_fadvise() will return -ENOSYS under 2.4.

    However... If you write any amount of data to a file with O_DIRECT, that
    will, as a side-effect, remove _all_ of that file's pagecache. In 2.4 as
    well as 2.6. So you could scrub the pagecache by reading the first 4k then
    writing it back with O_DIRECT.

    However O_DIRECT is supported on very few filesystems in 2.4. ext2 and
    reiserfs have it.

    XFS in 2.4 has O_DIRECT, I think, but I don't know if the invalidation
    side-effect works on XFS.

    > A related question...if no posix_fadvise() advice has been given, does
    > reading sequentially every byte of an 8GB file on a machine with <=
    > 8GB of RAM guarantee that any page cache data that existed on the
    > machine prior to the start of the 8GB read is now gone?

    It's not guaranteed that this will work - if the pages which you're trying
    to evict were accessed multiple times then it may take more page
    replacement to reliably shoot them down. But writing a 2xmemory file and
    then deleting it will be a reasonably effective way of evicting most of
    the other pagecache.

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Denis Vlasenko: "Re: [PATCH 2.6.8-rc2] intel8x0.c to include CK804 audio support"

    Relevant Pages

    • Re: [PATCH/RFC] A method for clearing out page cache
      ... > avoids the hackiness of both /proc and the kernel threads. ... syscall numbers are per-arch. ... >> much of the node's pagecache should be removed. ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: cache limit
      ... the problem occurred due to pagecache. ... >> response time get uneven, and maximum time became 10 seconds. ... The latest 2.4 or 2.5 standard kernel does not have such a parameter. ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: cache limit
      ... the problem occurred due to pagecache. ... > response time get uneven, and maximum time became 10 seconds. ... Please try the 2.4.18 based redhat kernel, ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: -mm merge plans for 2.6.23
      ... which the kernel even vaguely attempts to address, ... to be solved for mmapped files and even plain old pagecache. ... is a workload shift causing transition to state B, ... We would need to add a means by which userspace can repopulate ...
      (Linux-Kernel)
    • Re: cache limit
      ... >but the kernel is caching it at the expense of other programs. ... I also have the same problem about pagecache. ... extern unsigned long num_physpages; ... typedef int filler_t; ...
      (Linux-Kernel)