Re: Throughput question with CF/DiskOnChip

From: jro (john_at_jrobot.net)
Date: 01/04/05 

Date: 4 Jan 2005 10:39:55 -0800

Michael Schnell wrote:
> > Hmmm...I have not read about this. Of course I can understand the
> > possibility of losing a portion of a file that is in the process of
> > being flushed to disk if the CF is removed with the write in
progress
> > (and this is an acceptable constraint for this particular
application).
> > However, I don't understand how the CF card could be damaged and
made
> > unusable if it is removed at the wrong time. My PDA has a CF card,
and
> > it has no knowledge of when I would remove the CF card...but I can
> > remove the card whenever I want. How does it solve this problem?
> >
>
> This has been discussed several times in this forum. Please see the
> back-log. To handle wear out effects, a CF internally monitors write
and
> erase times and replaces dying blocks with fresh (spare) ones. To do
> this is uses some blocks as reference table. If such a table is
changed
> (which can happen within any a normal write request), part of it is
> erased and then rewritten. When power goes down in that moment the
table
> is damaged and the CF is unusable as it can't find the internal
memory
> block associated to an external address range. It can't even be
> formatted with normal means. There will be special propriety IDE
> commands that allow to revive the CF by rewriting the address
allocation
> table to a standard default allocation (loosing all data).

Thanks for the heads up on this. I went back through the archive and
found some of the references. In my current system, there is no way to
gracefully power down the board to ensure that the power to the CF card
is maintained for up to a minute after the last write (as you
suggested). I still don't totally understand this though. Its hard
for me to believe that if the cause of this issue is during an updating
of a flash block that takes too long to write, the manufacturerers
could write the new flash block with the new table; once it was
successfully written, they could update the "pointer" to the bad-block
table...thus if power ever went down, there was always a usable version
of the table. But maybe I'm oversimplifying the issue (or don't
totally understand it...). To mitigate this for now, I read that some
manufacturers of CF cards put super capacitors in their cards to
provide juice when needed if power is removed. I looked around and
couldn't confirm any manufacturer who does this. Does anyone know of
one? ST Micro was mentioned in the thread where I found this...
>
>
> >
> >
> > The ext2 files system on top of the DiskOnChip appears to be a
fairly
> > common activity:
> >
> > http://www.rtd.com/NEW_appnote/SWM-640000017%20rev%20A.pdf
> > http://www.gctglobal.com/Download/DiskOnChip/diskonchip.html
> > http://www.denx.de/twiki/bin/view/DULG/DiskOnChip
> >
> > Why exactly do you discourage this activity?
> >
> >
>
> I was speaking about a DOC with no hardware intelligence (like that
of a
> CF) but that is a quite naked flash chip.

I don't think a DiskOnChip exists without hardware intelligence...my
understanding is that the DOC utilizes something called TrueFFS, which
is a flash file system that provides wear-leveling etc. underneath any
normal file system being used (ext2 in this case). It provides an
interface that appears to be a standard block device to the OS, while
still providing this capability. See the following for a brief
overview (pg. 7 specifically):
http://www.m-systems.com/files/documentation/doc/TrueFFS_6.x_QRG_Rev1.3.pdf

So, I guess it seems like ext2 on top of TrueFFS won't cause any
burned-out flash cell issues. Am I wrong here? What am I missing?
>
> EXP2 and all "normal" file systems rewrite some blocks of the "disk"
> (e.g. the FAT with a DOS file system, more complex structures with
e.g.
> EXP2). Writing a single byte in a Flash causes a complete block (size

> depending on hardware 128 Byte ... 128 K) to be erased and rewritten.
A
> flash block can only be rewritten a certain number of times
(depending
> on hardware 10.000 ... 1.000.000 times). When using the chip for some

> kind of log file, this number can be reached quite fast. A dedicated
> flash file system knows about the flash blocks and (re)uses them
> cyclicly to manage this "wear out" effect. Moreover it will
> automatically remove blocks that get unwritable in spite of the
rotation
> scheme.

Ok, I understand here that you are referencing the flash block that
would be continually being updated, such as the block containing the
FAT in a normal DOS system. However, if this is on top of the TrueFFS,
it shouldn't be an issue (see above). Now if this is on a Compact
Flash card where there isn't a driver taking care of things like
wear-leveling, I can understand that this would be a problem. I've
looked around some and tried to understand what smarts commonly come in
a Compact Flash card, but there doesn't seem to be much out there (or
it is totally up to the manufacturers, as long as they provide the IDE
interface they need to). I've got to believe that people have had the
need to stream data to flash devices on the order of ~150 kbps before,
and that they were able to make it happen. I'm still playing around,
and will probably start doing a bunch of test with dd to dump /dev/zero
to a test file in different block sizes (bs=2k, bs=4k, etc) and see if
I can figure out the sweet spot as far as efficiency goes.
>
>
> > IIRC, JFFS2 was in the works for supporting the DOC, but at the
time
> > the system was being developed it was not ready yet.
>
> JFFS2 is ready since ages, but of course you need a media access
driver
> for the device in question, too. If that is available now and
specified
> for JFFS usage, this seems the way to go.
>
You can also check out this cool calculator for the DOC, which gives me
much better hope as far as wearing out the flash device:
http://www.m-sys.com/Content/Developer/Calc.asp

>
> > we haven't seen the
> > device fail (unless, of course, the "blip" we are seeing is due to
the
> > number of bad blocks increasing, and thus the DOC driver/hardware
> > taking longer and longer to find valid flash blocks to write to).
> >
>
> I don't think so. (There should be a way to read the count of bad
blocks
> from the media access driver.) I suppose the "blip" is just a cache
> write-back, that EXT2 does. This is not a good thing, either, as a
large
> cache will make you loose a lot of data when power fails and the file

> system is not dismounted before. Here a journaling file system could
> help a lot, but same creates even more write accesses and increases
the
> wear out problem.
>
> -Michael

Thanks for the input, and I'll be awaiting your collective responses
:-)

John

Relevant Pages

  • Why Spartan-3e is the best
    ... RAM based FPGA offerings). ... * can load not only from SPI Flash but also from Atmel Dataflash ... S3e and MMC Card socket can boot from the removabale flash media ... This is VERY good feature as it allows cheap Flash ROMs to be used for both ...
    (comp.arch.fpga)
  • Re: Throughput question with CF/DiskOnChip
    ... power is removed within this time the card may be damaged and unusable. ... A DOC should not be accessed with an EXT2 file system. ... Hopefully the DOC is just a Flash array accessible in memory blocks with ... Same should take care of wear out effects and supposedly will not do the ...
    (comp.os.linux.embedded)
  • Re: making a removable SSD drive nonremovable
    ... Using a FLASH based memory card in the way you suggest is a ... Putting in a FLASH based memory will make ... Are you certain that it's not battery backed RAM? ...
    (microsoft.public.windowsxp.hardware)
  • Re: making a removable SSD drive nonremovable
    ... Using a FLASH based memory card in the way you suggest is a very ... Are you certain that it's not battery backed RAM? ...
    (microsoft.public.windowsxp.hardware)
  • Re: making a removable SSD drive nonremovable
    ... Using a FLASH based memory card in the way you suggest is a very bad idea. ... Putting in a FLASH based memory will make Windows take a very significant performance hit. ... Are you certain that it's not battery backed RAM? ...
    (microsoft.public.windowsxp.hardware)