Re: A few questions before assembling Linux 7.5TB RAID 5 array

Conor <conor.turton@xxxxxxxxx> wrote:
I figured as much, but was hoping that someone else would say "Hey, in
my experience ___KB chunks are best for your situation,

Sadly, no as usage and typical filesize play a large part in it and no
two arrays are going to be used the same.

If the usage and file sizes are known that knowledge might be useful for
tuning performance when choosing chunk size. However, if some kind of
general purpose storage, which will host different filesizes, is built
wouldn't it still be possible to choose a good chunk size? In such a case
when it wouldn't be possible to optimize for the files stored on raid I
think that I instead would try to optimize the chunk size so it for sure
fits into the cache of each HD in the raid array.

If a chunk fits into the cache it will quickly be written to the fast
cache, then the next chunk will be written to the cache of the next drive
in the array and so on. Assuming we have enough drives and have them
configured as RAID5 or RAID0 we might be lucky and allways write to a
fast and ready cache only waiting to be filled. With RAID3 there will
always be writes to one single parity drive which will become a
bottleneck, this is why RAID3 almost never is used today.

regards Henrik
--
The address in the header is only to prevent spam. My real address is:
hc8(at)uthyres.com Examples of addresses which go to spammers:
root@xxxxxxxxxxxxx root@localhost

.

Relevant Pages

  • reshaping fread output of fortran unformatted binary file
    ... I need to convert it to matlab using fread ... I am unable to reshape the array to match the ascii ouput from the ... C Reads a small header before each chunk of data ...
    (comp.soft-sys.matlab)
  • Re: Audio programming - WAV data chunk format issues
    ... parse the file for the first "data" chunk, write this chunk it in the ... to the end of the array and so on... ... otherwise a (compiler dependend) packed array must be used, ...
    (comp.dsp)
  • Re: Best performance implementation?: Sharing data between threads
    ... If I am sending the whole data block in one chunk does it makes sense ... and send the contents of the array. ... ISTR thread::send must go through the string rep, while the TSV doc ... incurs less contention, ...
    (comp.lang.tcl)
  • Re: Getting The Binary Code Of File.
    ... and therefore I need to know the binary code of file. ... If the file is not too large then it is usually easier to get the entie file into one array, so that it is all in memory at the same time. ... Otherwise, for larger files, it might be better to get it in a number of separate "chunks" of perhaps a couple of megabytes each and deal with each chunk separately before loading the next chunk into memory. ... In this specific case the ReDim line sets up the dynamic array b so that it contains the same number of elements as their are bytes in the file. ...
    (microsoft.public.vb.general.discussion)
  • Re: Serial Port Communications Using Byte Array
    ... chunk by chunk using the byte array technique. ... Dim sChunckofData As String ...
    (microsoft.public.vb.controls)