Re: SATA-performance: Linux vs. FreeBSD

Am Dienstag, 13. Februar 2007 11:16 schrieben Sie:
Martin A. Fink wrote:
Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
Martin A. Fink wrote:
I have to store big amounts of data coming from 2 digital cameras to
disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
for
a long period of time. So it is important for me that the harddisk drive
is
reliable in the sense of "if it is capable of 50 MB/s then it should
operate
at this speed. Constantly."
The good old handful of suggestions:

- Use a dedicated disc for the task.

I used a dedicated disk for this task. No one else besides the task is
writing
to it!

OK.

- Use an empty disc so there is no fragmentation.

All tests were performed on empty disk!

OK.

- Buy a bigger disk, they have high bandwidths.

I have a flash disk from a manufacturer who grants me 48 MB/s. And FreeBSD
as
well as Windows reach this value. Only Linux 2.6.18 is far away from it
(42
MB/s)

Even 48MB/s is quite low.
I've reached up to 70MB/s with a single 500GB Seagate model and even my
older HDDs all reach 60MB/s (at least on the outer cylinders)
But i haven't tested any "sync/fsync" in between, only after.

Please Read Carefully! I talk about flash disk, not normal harddisks. There
are no mechanical parts in flash disks, only flash memory. And therefore
48MB/s is excellent (compared to all other available disks)


- Buy a more "specialized" disc.

see above

for e.x.: Western Digital Raptor X(*) a 150GB, 10-KRPM S-ATA disc.
- Buy several discs and use RAID 0
or alternate between discs when writing.

What I have to build is an application for the International Space Station
ISS. I am limited with power and space. So If the disk is able to write
constantly 48 MB/s then the Operating System should do this!

OK. That appears to be a serious constraint.
Do HDDs cope well with zero gravity?

Yes and no. Yes: standard desktop HDDs are unproblematic. Laptop HDDs have
g-force shock hardware that works on zero-g detection and thus Laptop HDDs
can't be used in space. At least modern ones can't...

At least the SSD won't have a problem with that. ;-)

The problem is: FreeBSD is fast, but lacks of some special drivers. Linux
has
all drivers but access to harddisk is unpredictable and thus unreliable!
What can I do??

Personally i haven't had such bad write speeds in years. Taking USB
connected and/or encrypted partitions aside.
But on the other hand: I don't sync(fsync) until i have to.

If you don't have to - no problem. But if you use filesystem you do a fsync
every time you close the file (and filesize is less then 1-2 GB)
And personally i have good (and constant bandwidth) experience using XFS as
a filesystem.
(I have 41 HDDs with a total capacity of 10.5 TB, performance is quite
important for me.)

Also you have skipped the information how the images "arrive" on the system
(PCI(e) card?), that may be important for an "end to end" view of the
problem.

Images arrive via Gigabit Ethernet. GigE Vision standard. (PCIe x4)

And what's also missing. What is "a long period of time".
Calculating best-case with the SSD:
27GB divided by 30MB/s only gives a bit more than 15 Minutes.
And worst case with 50MB/s is less than 10 Minutes.

Well. The testdrive has 27GB. The final drive will have 225 GB. And there will
be 3 cameras and thus 3 disks. This means we talk about 140 MB/s for around
90 minutes.
For space applications with low power but high performance this is a long
time... ;-)





--
Real Programmers consider "what you see is what you get" to be just as
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated,
cryptic, powerful, unforgiving, dangerous.



--
Dipl. Physiker
Martin Anton Fink
Max Planck Institute for extraterrestrial Physics
Giessenbachstrasse
85741 Garching
Germany
Tel. +49-(0)89-30000-3645
Fax. +49-(0)89-30000-3569
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: EWF and Remote Boot
    ... At this time EWF did not works correct. ... It does not make any sense why you see new disk after just booting from network and them booting from flash disk. ... > My remote boot image does not include EWF components, ...
    (microsoft.public.windowsxp.embedded)
  • Re: Motion LS800 table HD failure at 14,000 feet
    ... swapfile, and registry, to the SD card. ... What we really need to do is boot the OS from a flash disk and run it in a ramdisk. ... I am not a Windows guy, so I don't even know if that's possible without paying Micro$oft lots of money for its development kits or embedded version, but common Linux "Live" CDs such as Ubuntu run like that, so the basic premise of operation is sound. ... In fact, I just built a custom, full-featured Linux distribution that takes up only 400MB on a flash disk and I wasn't trying to save space. ...
    (rec.aviation.ifr)
  • Re: Safely remove usb flash drive - Can someone explain?
    ... Clicking the Safe Remove icon ensures that all pending ... disk writes are completed before you're given the go-ahead. ... If you just pull it out then you might corrupt the contents ... then remove the flash disk immediately after the process ...
    (microsoft.public.windowsxp.hardware)
  • Re: getting usb mass storage to finish before running init?
    ... I realized that in my config only the flash disk has 2 partitions (the ... >> well, the root filesystem is an initrd, so I can't do that. ...
    (Linux-Kernel)
  • Re: CD Burning? media and burner questions
    ... CDR and DVDR discs without a single return from a customer ... Note that many brands available on retail shelves are only ... There are some burning programs and disk utilities that will ...
    (rec.audio.pro)