poor disk IO when copying file cross partitions on RedHat 3.0 EL

From: Jieming Wang (jieming_wang_at_hotmail.com)
Date: 10/01/04 

Date: 30 Sep 2004 19:10:18 -0700

We have a strange prolem. The IO performance tests showed that when
copying a large file cross parition, the transfer rate is almost 10
times slower than copying between the partition.

# ls -ltr /home/test/junk
-rw-r--r-- 1 root root 1152981025 Sep 30 15:16
/home/test/junk

[root@customer2 test]# df -k /home/test /var/test
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda7 4370688 2463220 1685444 60% /home
/dev/sda6 4370688 130728 4017936 4% /var

===copy cross paritions===
# time cp /home/test/junk /var/test
real 5m50.711s
user 0m0.190s
sys 0m15.380s

===copy on the same parition===
# time cp /home/test/junk /home/test/junk.bak
real 0m38.867s
user 0m0.100s
sys 0m9.210s

It seems that the problem only happens with ips SCSI host adapter
(alias scsi_hostadapter ips) which is used by IBM PCI ServerRAID.

Here is the output from lsmod:
...
ips 45156 7
sd_mod 13712 14
scsi_mod 114344 2 [ips sd_mod]

===sample output from dmesg===
SCSI subsystem driver Revision: 1.00
scsi0 : IBM PCI ServeRAID 7.00.15 Build 625 <ServeRAID 6i>
  Vendor: IBM Model: SERVERAID Rev: 1.00
  Type: Direct-Access ANSI SCSI revision: 02
  Vendor: IBM Model: SERVERAID Rev: 1.00
  Type: Processor ANSI SCSI revision: 02
  Vendor: IBM Model: 32P0032a S320 1 Rev: 1
  Type: Processor ANSI SCSI revision: 02
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
Partition check:
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 sda8 >
Journalled Block Device driver loaded
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.