Re: updatedb for very large filesystems



WEll, I am more interesting is searching a large Networked filesystem.



On Thu, Oct 2, 2008 at 6:18 AM, Ron Johnson <ron.l.johnson@xxxxxxx> wrote:
On 10/02/08 04:28, James Youngman wrote:

On Wed, Oct 1, 2008 at 12:15 PM, Mag Gam <magawake@xxxxxxxxx> wrote:

I was wondering if its possible to run updatedb on a very large
filesystem (6 TB). Has anyone done this before? I plan on running this
on a weekly basis, but I was wondering if updatedb was faster than a
simple 'find'. Are there any optimizations in 'updatedb' ?

With findutils you can update several parts of the directory tree in
parallel, or update various parts on a different time schedule.

Here's an example with three directory trees searched in parallel with
one being searched remotely on another server and then combined with a
canned list of files from a part of the filesystem that never changes.

find /usr -print0 > /var/tmp/usr.files0 &
find /var -print0 > /var/tmp/var.files0 &
find /home -print0 > /var/tmp/home.files0 &
ssh nfs-server 'find /srv -print0' > /var/tmp/srv.files0 &
wait

Since find is so disk-intensive, isn't this is only of benefit if /usr, /var
and /home are on different devices?

sort -f -z /var/tmp/archived-stuff.files.0 /var/tmp/usr.files0
/var/tmp/var.files0 /var/tmp/home.files0 /var/tmp/srv.files0 |
/usr/lib/locate/frcode -0 > /var/tmp/locatedb.new
rm -f /var/tmp/usr.files0 /var/tmp/var.files0 /var/tmp/home.files0
/var/tmp/srv.files0

cp /var/cache/locate/locatedb /var/cache/locate/locatedb.old
mv /var/tmp/locatedb.new /var/cache/locate/locatedb




--
Ron Johnson, Jr.
Jefferson LA USA

"Do not bite at the bait of pleasure till you know there is no
hook beneath it." -- Thomas Jefferson


--
To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx with a subject
of "unsubscribe". Trouble? Contact listmaster@xxxxxxxxxxxxxxxx




--
To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx
with a subject of "unsubscribe". Trouble? Contact listmaster@xxxxxxxxxxxxxxxx



Relevant Pages

  • Re: updatedb for very large filesystems
    ... not the size of the filesystem). ... Are there any optimizations in 'updatedb'? ... The implementation of updatedb mlocate package has an important ... To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx ...
    (Debian-User)
  • Re: updatedb for very large filesystems
    ... Are there any optimizations in 'updatedb'? ... canned list of files from a part of the filesystem that never changes. ... -- Thomas Jefferson ... To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx with a subject of "unsubscribe". ...
    (Debian-User)
  • Re: updatedb for very large filesystems
    ... not the size of the filesystem). ... Are there any optimizations in 'updatedb'? ... The implementation of updatedb mlocate package has an important ... To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx ...
    (Debian-User)
  • Re: updatedb for very large filesystems
    ... Are there any optimizations in 'updatedb'? ... With findutils you can update several parts of the directory tree in ... canned list of files from a part of the filesystem that never changes. ... To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx ...
    (Debian-User)
  • Re: -mm merge plans for 2.6.23
    ... > I'd just like updatedb to amortize its work better. ... > accurate index on the filesystem. ... http://sourceforge.net/projects/rlocate/ - Source Forge Project Summary ...
    (Linux-Kernel)