Re: Linux file copy slowness ??

parvinderb@xxxxxxxxx writes:
>Hi,
>
>We are trying to copy a 15Gig file from one directory to another
>(inside same partition) for example:
>
>cp /data/BigFile /data/data/ (Bigfile - being file that is greater
>than 15 gig).

So, both files are on the same disk, and you're getting 7.8Mb/second.

For reference, this is the output from a 1gb random transfer on
a 36MB 10krpm seagate cheetah:

$ t2 -r -f /dev/sday -b 131072 -i 1000
Device /dev/sday has 70022843 512-byte blocks
131072000 bytes (1000 131072-byte blocks to 1 devs) in 14.50 seconds (9039.45 Kbytes/sec)

(-r accesses the device randomly).

Drop the block size, and:
$ t2 -r -f /dev/sday -b 4096 -i 1000
Device /dev/sday has 70022843 512-byte blocks
4096000 bytes (1000 4096-byte blocks to 1 devs) in 9.00 seconds (455.11 Kbytes/sec)

So, your 7.8MB/second seems pretty good. You may get better
performance if you mk2efs your filesystem with -b 8196 instead
of 4096.

$ stat -f /
File: "/"
ID: 0 Namelen: 255 Type: ext2/ext3
Blocks: Total: 10078499 Free: 9590416 Available: 9078445 Size: 4096
Inodes: Total: 5128192 Free: 5065884

(the block size is the "Size:" field).

-----------Just for reference:

Pure sequential access on a Cheetah:
$ t2 -f /dev/sday -b 131072 -i 1000
Device /dev/sday has 70022843 512-byte blocks
131072000 bytes (1000 131072-byte blocks to 1 devs) in 3.50 seconds (37449.14 Kbytes/sec)

And, the advantages of fibrechannel and lots of JBOD (and a 2Gbit switch):

$ t2 -f /dev/sdag -f /dev/sdah -f /dev/sdam -f /dev/sdan -f /dev/sdaq -f /dev/sdap -f /dev/sdaa -f /dev/sdab -b 131072 -i 10000
Device /dev/sdag has 70022843 512-byte blocks
Device /dev/sdah has 70022843 512-byte blocks
Device /dev/sdam has 70022843 512-byte blocks
Device /dev/sdan has 70022843 512-byte blocks
Device /dev/sdaq has 163397632 512-byte blocks
Device /dev/sdap has 163397632 512-byte blocks
Device /dev/sdaa has 163397632 512-byte blocks
Device /dev/sdab has 163397632 512-byte blocks
10485760000 bytes (10000 131072-byte blocks to 8 devs) in 52.00 seconds (201649.23 Kbytes/sec)


Note that the test program, t2, uses O_DIRECT to bypass the operating system
file cache, and uses io_submit(2) fpr asynchronous I/O.

scott
.