Oracle Low performance - IO bottleneck with raw devices

Hello,

we've RedHat Linux AS3 (upd. 1) running in a 8 proccessor IA64 Itanium
II machine with RAM-16Gb. Oracle 9i under RawDevices with an oracle
blocksize of 8k and 2 QLogic Fibre Channel Adapters (QLA2340) with
Symmetrix EMC disk array. We also have ASync I/O activated in oracle.

Look at this:
--------------------------------------------------------------------------------------------------------------
# sar (copy-paste a little fragment but all day is similar)
00:00:00 CPU %user %nice %system %iowait %idle
........
10:05:00 all 29,54 0,00 6,14 27,96 36,36
10:10:00 all 46,80 0,00 5,82 15,32 32,06
10:15:00 all 30,88 0,00 2,96 17,90 48,25
10:20:00 all 32,21 0,00 7,69 19,01 41,09
10:25:00 all 37,14 0,00 6,27 38,62 17,97
10:30:00 all 33,94 0,00 7,20 29,62 29,24
10:35:00 all 52,47 0,00 10,31 28,08 9,15
10:40:00 all 55,75 0,00 5,87 13,78 24,60
10:45:00 all 26,06 0,00 7,99 12,23 53,72
--------------------------------------------------------------------------------------------------------------
# iostat 2 /dev/sdc1 (there are many disks but this is mainly used)

cpu-med: %user %nice %sys %iowait %idle
13,02 0,00 0,40 1,85 84,73

Device: tps Blq_leid/s Blq_escr/s Blq_leid Blq_escr
sdc1 1534,25 2673,31 174,09 5344 348

cpu-med: %user %nice %sys %iowait %idle
13,20 0,00 1,52 11,42 73,86

Device: tps Blq_leid/s Blq_escr/s Blq_leid Blq_escr
sdc1 8482,14 15847,74 125,06 31680 250
--------------------------------------------------------------------------------------------------------------
# iostat -x 2 /dev/sdc1

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s
avgrq-sz avgqu-sz await svctm %util
sdc1 6833,34 114,06 7902,86 95,05 14743,20 209,10 7371,60
104,55 1,87 34,12 4,26 0,12 93,15

cpu-med: %user %nice %sys %iowait %idle
15,23 0,00 2,11 11,32 71,34

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s
avgrq-sz avgqu-sz await svctm %util
sdc1 7149,99 25,01 8151,98 35,52 15295,47 134,07 7647,73
67,03 1,88 32,95 4,03 0,11 92,75
--------------------------------------------------------------------------------------------------------------
I´ve only shown /dev/sdc statistics because almost ALL the RAWs are in
that device.

Well, now the questions (¡¡of course!! ;-) ):

1.- There is a generic low performance at the system, at SO level,
at oracle level and at application level. ¿Can we suppose that there
is an IO bottleneck?
2.- ¿Are the "wrqm/s" values correct?

We've different DDBB with similar characteristics running under similar
hardware but with EXT3 and we've got much better performance.
We also have RAW DEVICES under AIX (4.3 and 5.2) and the performance is
PERFECT.

I've been reading different manuals and documentation about Raw Devices
and Direct I/O in Linux. Everything makes me think that oracle
blocksize (8K) should not affect performance drastically.

I don´t know how to solve this problem. Do you think that an oracle
db_block_size parameter of 16K should help???

We've planned move some RAWs to diferent PVs to distribute the load of
de IO but I'm not sure that is the only problem ....

Maybe this message is OFF-TOPIC but I think it doesn´t, sorry me if it
is.

Some help, please please please ....

Thanks in advance.

.

Relevant Pages

  • RE: Oracle on Red Hat best practices
    ... Unless you are implementing Oracle with ASM, which uses raw devices. ... Backups are most critical, ...
    (RedHat)
  • RE: Oracle on RHEL4 with ASM problem
    ... bounces@xxxxxxxxxx] On Behalf Of Tom Callahan ... Oracle on RHEL4 with ASM problem ... ASMLIB for Linux may not support System-based adding of raw devices, ... "When installed on raw devices or, as recommended by Oracle, on block ...
    (RedHat)
  • Re: Oracle 9i on Solaris 8 w/ VxVM - UFS logging?
    ... >> and data files on raw devices whenever possible. ... >> For all other Oracle files, like Oracle binaries themself or control files, ... Connor McDonald ... "GIVE a man a fish and he will eat for a day. ...
    (comp.unix.solaris)
  • Re: ASM Question - Best Practice
    ... I disagree with Frank. ... I haven't had any problems with ASM and I find it much more convenient then raw devices. ... My opinion is that for almost everything Oracle ... ...
    (comp.databases.oracle.server)
  • Re: Oracle on RHEL4 with ASM problem
    ... exising lines for raw devices, ... # End Oracle 10g Devices ... You can add rawdevices to that list as well. ... then "service rawdevices start" (remember to chkconfig that service to ...
    (RedHat)