Re: Max no of files in a directory

Smeeta staggered into the Black Sun and said:
On May 30, 10:35 pm, Dances With Crows wrote:
Smeeta staggered into the Black Sun and said:
I want to find out the max number of files possible in a directory.
On which filesystem? If ReiserFS, which hash is being used? The
answers to these questions will determine the answer to your
question.
ext3

/usr/src/linux/Documentation/filesystems/ext2.txt :
------------
Using a hashed directory index (under development) allows 100k-1M+ files
in a single directory without performance problems (although RAM size
becomes an issue at this point).

The (meaningless) absolute upper limit of files in a single directory
(imposed by the file size, the realistic limit is obviously much less)
is over 130 trillion files. It would be higher except there are not
enough 4-character names to make up unique directory entries, so they
have to be 8 character filenames, even then we are fairly close to
running out of unique filenames.
------------

Hashed dir indexes are present by default now. I know you can fit at
least 50,000 files in one dir on ext3 without serious performance
problems because I've done it. On an ext3 filesystem, the absolute
limit is given by the number of inodes on the filesystem, which is
probably less than 130*10^12.

However, if you're going to be putting tens of thousands of files in one
dir, you should think about ways to split those files up into subdirs.
That will make it easier to deal when you have to deal with millions of
files instead of tens of thousands.

--
The squirrels now live in fear of the magic exploding compost heap.
--Dan Holdsworth
My blog and resume: http://crow202.dyndns.org:8080/wordpress/
Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
.

Relevant Pages

  • Re: [patch] ext2/3: document conditions when reliable operation is possible
    ... of your ext3 + flash card issue - is it the ftl stuff doing out of order ... and ext3 can't cope with that. ... the problem is that people have been preaching that journaling filesystems eliminate all data loss for no cost. ... depending on how much data gets lost, you may or may not be able to recover enough to continue to use the filesystem, and when your block device takes actions in larger chunks than the filesystem asked it to, it's very possible for seemingly unrelated data to be lost as well. ...
    (Linux-Kernel)
  • Re: [patch] ext2/3: document conditions when reliable operation is possible
    ... of your ext3 + flash card issue - is it the ftl stuff doing out of order IO's? ... The problem is that flash cards destroy whole erase block on unplug, ... this isn't a filesystem specific cliam; ...
    (Linux-Kernel)
  • Re: [patch] ext2/3: document conditions when reliable operation is possible
    ... wouldn't it be an ext3 bug? ... many naive filesystem designers make is that writes succeed or they ... That's just the way hardware works. ... netbook would cause the flash card to become disconnected from the ...
    (Linux-Kernel)
  • Re: How git affects kernel.org performance
    ... On Sat, 6 Jan 2007, H. Peter Anvin wrote: ... almost no other common system call has to (99% of all filesystem calls can ... directory, it serializes getdents. ... we had the same problem on a file server with ext3. ...
    (Linux-Kernel)
  • Re: [Ext2-devel] [RFC 0/13] extents and 48bit ext3
    ... res digital format across your entire life in near-line format. ... I'd think the quickest path is to relieve the pressure now in ext3. ... "Linux 2.5". ... I acknowledge the concerns about filesystem stability and Linus's points about improperly shared code. ...
    (Linux-Kernel)