Re: tar block question

On Sun, Sep 14, 2008 at 10:03 AM, Mag Gam <magawake@xxxxxxxxx> wrote:
I have to tar up many small files. I have over 30k files in a
directory. What is the best way to do this? If I tar it its taking a
long time but from what I have been reading if I increase my blocksize

Just did a test on a similar-sized directory ( my old mail dir, about
30K in small files). Adding a blocking factor does not seem to speed
things up that quickly, and I think I'm getting the speed up from the
disk cache, although I did a sync to see if that would make a
difference. Still, I get a little bit of a speed up, but I don't think
it is very significant. Also, IIRC, blocking factors are mostly for
tapes rather than regular files. System is a 64bit dual core AMD with
ubuntu hardy with all updates applied, and the disk is a 500 gb
western digital sata/3.0 gbps model.

Here's a (few) test runs:

fox@newbox:~/.Mail/temp$ time tar -cf foo.tar ~/.Mail/temp/inbox/*
tar: Removing leading `/' from member names

real 0m7.736s
user 0m1.412s
sys 0m1.516s
dfox@newbox:~/.Mail/temp$ ls -l foo.tar
-rw-r--r-- 1 dfox dfox 166932480 2008-09-14 12:10 foo.tar
dfox@newbox:~/.Mail/temp$ rm -f foo.tar
dfox@newbox:~/.Mail/temp$ time tar -cf foo.tar -b 200 ~/.Mail/temp/inbox/*
tar: Removing leading `/' from member names

real 0m5.786s
user 0m1.396s
sys 0m1.544s
dfox@newbox:~/.Mail/temp$ time tar -cf foo.tar -b 200 ~/.Mail/temp/inbox/*

dfox@newbox:~/.Mail/temp$ ls -l foo*
-rw-r--r-- 1 dfox dfox 167014400 2008-09-14 12:11 foo.tar
dfox@newbox:~/.Mail/temp$ rm -f foo.tar
dfox@newbox:~/.Mail/temp$ sync
dfox@newbox:~/.Mail/temp$ time tar -cf foo.tar -b 2000 ~/.Mail/temp/inbox/*
tar: Removing leading `/' from member names

real 0m3.675s
user 0m1.248s
sys 0m0.852s
dfox@newbox:~/.Mail/temp$ ls inbox/* | wc -l
30269


--
To UNSUBSCRIBE, email to debian-user-REQUEST@xxxxxxxxxxxxxxxx
with a subject of "unsubscribe". Trouble? Contact listmaster@xxxxxxxxxxxxxxxx

Relevant Pages

  • Fwd: [Cron <root@mail> sh /root/save_slox]
    ... Shutting down fetchd/sbin/rcfetchd: line 76: killproc: command not found ... tar: Removing leading `/' from member names ...
    (RedHat)
  • errors on pkg_create for p5-* ports
    ... After installing 7.1R ports I'm trying to create packages ... Removing leading '/' from member names ... tar: Error exit delayed from previous errors. ...
    (freebsd-questions)
  • Re: (tar -cf - /etc|gzip; dd if=/dev/zero count=1)...|rsh foo dd of=/dev/st0
    ... Removing leading `/' from member names ... multiple of 512, the last write to /dev/st0 will not be 512 ... I see no tar related errors here: ...
    (comp.unix.shell)
  • Re: .sh & tar
    ... Removing leading '/' from member names ... I have tar outputting to> /dev/null and still get this message. ... If you want to remove the REASON for "tar: Removing leading '/' ... you need to re-create the archives and ...
    (freebsd-questions)
  • Re: .sh & tar
    ... Removing leading '/' from member names ... I have tar outputting to> /dev/null and still get this message. ... If you want to remove the REASON for "tar: Removing leading '/' ... you need to re-create the archives and ...
    (freebsd-questions)