Uncached stat performace [ Was: Re: Kernel SCM saga.. ]

From: Ragnar Kjørstad (kernel_at_ragnark.vestdata.no)
Date: 04/08/05

  • Next message: Chris Wedgwood: "Re: Uncached stat performace [ Was: Re: Kernel SCM saga.. ]" 
    Date:	Fri, 8 Apr 2005 22:11:51 +0200
To: Linus Torvalds <torvalds@osdl.org>

    On Fri, Apr 08, 2005 at 12:39:26PM -0700, Linus Torvalds wrote:
    > One of the reasons I do inode numbers in the "index" file (apart from
    > checking that the inode hasn't changed) is in fact that "stat()" is damn
    > slow if it causes seeks. Since your stat loop is entirely
    >
    > You can optimize your stat() patterns on traditional unix-like filesystems
    > by just sorting the stats by inode number (since the inode number is
    > historically a special index into the inode table - even when filesystems
    > distribute the inodes over several tables, sorting will generally do the
    > right thing from a seek perspective). It's a disgusting hack, but it
    > literally gets you orders-of-magnitude performance improvments in many
    > real-life cases.

    It does, so why isn't there a way to do this without the disgusting
    hack? (Your words, not mine :) )

    E.g, wouldn't a aio_stat() allow simular or better speedups in a way
    that doesn't depend on ext2/3 internals?

    I bet it would make a significant difference from things like "ls -l" in
    large uncached directories and imap-servers with maildir? 

    -- 
Ragnar Kjørstad
Software Engineer
Scali - http://www.scali.com
Scaling the Linux Datacenter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
  • Next message: Chris Wedgwood: "Re: Uncached stat performace [ Was: Re: Kernel SCM saga.. ]"

    Relevant Pages

    • Re: [PATCH] pipefs unique inode numbers
      ... do a stat on this inode, it would likely generate an EOVERFLOW error since ... due to this bug some apps complain about EOVERFLOW returned by stat() ... I also observed similar bug on Debian 3.0 which has 'find' compiled w/o FILE_OFFSETS=64, ... but NFS partition used 64bit inode numbers. ...
      (Linux-Kernel)
    • Re: Incident investigation methodologies
      ... > what's on the hard drive of a unix machine no matter what the rootkit may ... relied on things like readdirand stat() returning the information requested. ... void generic_fillattr(struct inode *inode, struct kstat *stat) ... void malicious_fillattr(struct inode *inode, struct kstat *stat) ...
      (Incidents)
    • Re: Weird problem with special characters in filename
      ... the path lookup of some of the entries fails for stat. ... If it ie a resiserfs it may have any number of strange properties! ... files are stored in the space after the dirents in the dir inode). ... You ned to fix it. ...
      (comp.os.linux.misc)
    • Re: Kernel SCM saga..
      ... > Doing the minimal stat cold-cache here is about 6s for local disk. ... One of the reasons I do inode numbers in the "index" file (apart from ...
      (Linux-Kernel)
    • Re: Big time difference between (unix) find and DIRECTORY : why?
      ... >> it's a directory you stat, ... >> leaf-node information comes in. ... > and the inode number, the inode contains everything else (but not, ... directory (do not care about any other properties) and you can /deduce/ ...
      (comp.lang.lisp)