datatransfer slow down with 1TB files

From: Christoph Klocker (cklocker_at_kth.se)
Date: 09/30/03

  • Next message: Joerg Schilling: "Re: Kernel includefile bug not fixed after a year :-(" 
    Date:	Tue, 30 Sep 2003 13:39:23 +0200
To: linux-kernel@vger.kernel.org

    Hi
    I am developing a system to stream large uncompressed videofiles which
    are about 1,5TB.
    I need a bandwith of 195MB/s (HD-SDI)
    my system is now:
    RH8 - kernel: 2.4.18
    xeon 2,4GHz
    1GB RAM
    3ware escalade controller 12 s-ata harddisks (seagate barracuda 7200.7 -
    120GB - SCSI)

    when I do different tests with bonnie++ I get very good results for a
    2GB file, sequential input 236mb/s,
    but when the files get larger the speed is going down significantly.
    at 10GB - 216MB/s
    at 100GB - 187MB/s
    at 1TB - 161MB/s
    the sequential output stays at 183MB/s any time, up to 1TB.
    As my harddisks have a sustained rate of 32-50 MB/s each it should not
    be the
    case that they influence the results. I also tested the raid full up to
    1,2TB and there was no difference in the results.
    Best filesystem performance I tested was on XFS or EXT2
    I applied min-readahead to 128 and max-readahead to 256

    I tested also with lmdd with the O_DIRECT option, and get lower results.
    lmdd if=internal of=fred bs=2m count=1000 fsync=1 direct=1
    2097.1520 MB in 12.5588 secs, 166.9860 MB/sec

    the driver for the videocard I use don't support 2.6
    please reply personally
    thanks
    christoph

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  • Next message: Joerg Schilling: "Re: Kernel includefile bug not fixed after a year :-("