Re: Data transfer for real time analysis

From: Paul Carpenter (paul$_at_pcserv.demon.co.uk)
Date: 04/29/04 

Date: Thu, 29 Apr 2004 21:52:21 +0000 (UTC)

On Thursday, in article <c6rn9d$laf$1@news.iucc.ac.il>
     pinit@ee.bgu.ac.il "Pini" wrote:

>Each sub unit has 60 channels. Each channel is sampled at 25KHz with 16 Bit
>resolution. Each sub unit produces 60*25000*16=24Mbit/sec.
>The data is arranged in frames of 25 samples. Each frame is about 1msec
>duration.
>The recorded data should be clustered into various categories. Based on the
>clustering the PC will activate some actuators.
>The data must be transferred to the PC for online storage and analysis. The
>analysis can be done on a DSP board but the raw data still will be
>transferred to the PC for storage and performing fusion of the DSP analysis
>of each sub unit.

You still not answered half of the questions I laid out below and the above
adds more:-

1/ For many minutes/hours/days/weeks is this experiment to run?
        With the cost of memory these days having 1GB+ buffers in
        each sub-unit, or a system dealing with one or more of the
        sub-units will make storage for a run or buffering of the run
        possible.
2/ Do you really need all the data recorded?
3/ Does the analysis perform part of the control loop?
    So far you have said some of the clustering [of the recorded data]
    causes some unknown number of actuators to be activated.

4/ Does the storage and control have to be on the same unit?
    Separating the storage and analysis (two boxes or more receiving the same
    data) will help your I/O bottlenecks.
5/ With previous posts suggesting 4 - 12 subunits that is upto 720 channels
    of raw data, and some unspecified analysis data and no doubt some saving
    of actuator state data. This means UP TO 720*25000*16bit = 36,000,000 Bytes
    per second of storage without the overheads of disk file system and track
    seeking times to cope with. If the analysis produces at least two more
    channels of information, that is a lot of data being transferred over
    a common bus as well as the incoming datastreams. I don't know what
    bus you are expecting your PC to have to handle this data rate as most
    PCI slots are or the order of 300MB/s you are using up the bandwidth
    very quickly.
6/ Are the actuators related to the sub-units?
    I.E. is there 'n' actuators associated with each sub-unit or does the
    data from more than one sub-unit effect multiple actuators, or are the
    actuators affected by more than one sub-unit?

    If one group of data affects only one set of actuators, split the
    design down and deal with the data to storage either after the
    experiment has run or buffered by a large amount of memory in each
    sub-unit.

    If one DSP analysis only affects one set of actuators split this out
    into the sub-unit.

    If the analysis does NOT affect the control loop, consider either
    doing this post analysis or on a separate box.

Your problem here is NOT just the transfer of data from sub-units to one PC
it is all the data transfers in all directions trying to be done in one PC.
All of the data transfers involve overhead which you appear to have
overlooked.

You will need to either time stamp or sequence stamp various parts of your
data to match it all up, especially from the analysis to combine with
raw data. So splitting the functions and LARGE buffering of data is less
of a headache than trying to squeeze it all into one box. Splitting functions
makes control loop time EASIER to achieve.

This sort of data throughput really requires each sub-unit to have its
own dedicated high speed point-to-point link, to guarantee data throughput,
without clashes of different units on one link. Realistically these kinds
of point-to-point links may well be better served by _avoiding_ using
standard PC interfaces, as the non-deterministic nature of overheads and
the environment make it impossible to guarantee your control loop time
by putting everything in one box! Especially if a task is late or a frame
is corrupted and needs resending or an error like a disk bad block is
encountered.

How much is your hardware budget for all of this?
How much already exists?
What timescale are you looking at?

>??"Paul Carpenter" <paul$@pcserv.demon.co.uk> ??? ??????
>news:20040428.2227.298804snz@pcserv.demon.co.uk...
>> On Wednesday, in article <c6mjmn$r5j$1@news.iucc.ac.il>
>> pinit@ee.bgu.ac.il "Pini" wrote:
>>
>> >The system is developed for academic research of biological signals.
>> >
>> >Each channel sampling rate is 25KHz. Each data frame holds 25 samples.
>The
>> >data will be processed in frames.
>>
>> Considering like others you mentioned 25MHz before, I don't really know
>> what you are really trying to achieve.
>>
>> How many bits in a sample?
>> How many channels?
>> How long do you really need to record all this data for analysis?
>> Is the analysis the control loop, or post analysis to confirm the
>> control loop actions?
>> What is the analysis output - a decision, vector, matrix, extra
>> data stream to store?
>>
>> >I' planning to use a PC with 4 CPU's. One fore handling the data
>> >communication and memory allocation, one for handling the HD storage and
>the
>> >2 last CPU will be for number crunching. A second alternative is to use a
>> >PCI board with multiple DSP for data analysis and the PC will do the data
>> >fusion from all the channels for closing the loop (and data storage).
>>
>> Why put all the processing and storage in ONE BOX?
>> As others have said they all share the same datapaths,s o that becomes the
>> bottleneck.
>>
>> >We can use FPGA at the sub-units if required.
>> >We can not afford any data package loss.
>>
>> Why not store the data there for download after what is likely to be a
>burst
>> of data.
>>
>> >Regarding the selection of the communication protocol:
>>
>> As I doubt that you really want an hours recording of for example
>>
>> 4 channels of 8 bits = 32bits = 4 bytes
>> sampled at 25KHz = 100,000 Bytes/second
>>
>> For an hour = 3600 x 100,000 = 360,000,000
>Bytes/hour
>> = just over 343MB!
>>
>> This does not include saving of data analysis.
>>
>> Double it for 12/16 bit sample widths, add suitable multiplying factors
>> for increase in channels or longer data storage times.
>>
>> >TCP/IP:
>> >What will be worst the latency of each channel if I use TCP/IP Hub
>> >(100MBit/sec
>> >or 1000MBit/sec)?
>> >Will the use of windows CE guaranty the Ethernet frame overall latency?
>> >(Did someone tried the real-time TCP/IP extension from:
>> >http://www.vci.com/products/windows_embedded/rttcpip.asp )
>>
>> I would only consider this on data sets of this size when using:-
>>
>> a) burst capture, then store later
>> b) multiple connections on separated network segments
>> to achieve the packet rebuilding
>>
>> >I2C:
>> >What HW should I use on the PC side?
>>
>> Practically none as STANDARD controllers for PC use the slow bus speed of
>> 100KHz, whereas most 400KHz and 2MHz bus designs are usually from FPGA
>> or dedicated controller designs with short signal length within a
>subsystem.
>>
>> >What about SMBus? What should I use on the PC side?
>>
>> >USB:
>> >I can use the USB protocol but I have a latency of 1msec (can I reduce
>this
>> >latency)?
>> >What is the USB Hub latency?
>>
>> Depends on the USB configuration and how many layers you have and the
>> software controlling and number of devices on the bus.
>>
>> Personally I cannot see that the data analysis of this data set size
>> realistically being used in a ONE BOX situation.
>>
>> If the analysis is post experiment run, I would consider each unit having
>> its own large memory, using some form of seperate communications for the
>> control linking, and after experiment run download the data sets to the
>> storage and analysis unit. This method is the basic method of high speed
>> video capture to ensure no missing picture frames in a sequence.
>>
>> If the analysis is part of the control loop put DSPs in each sub unit
>> to process the data, then store original data and results in large memory
>> for later download. Still communicate results from frames/analysis to
>either
>> a central unit, with a reduced data set or analysis decision results.
>>
>> I think you need to revisit actually what you are trying to achieve,
>> what you actually need and when and where it is needed.
>>
>> The current outline looks like a wish list for an ideal world, onto
>> a single PC as that is available easily, but may not be the ideal
>> solution.
>>
>> >??"Pini" <pinit@ee.bgu.ac.il> ??? ??????
>> >news:c6h98p$3jm$1@news.iucc.ac.il...
>> >> hi,
>> >>
>> >> We are planning a data acquisition system with 4-12 sub units. Each sub
>> >unit
>> >> produce data of 25MBit/sec. The distance between the boxes and computer
>is
>> >> about 2 meters.
>> >> I'm willing to transfer and analyze the data in real-time. I'm willing
>to
>> >> close a control loop in 1-5msec (I. wish 100usec ......).
>> >> What is the recommended serial protocol?
>> >> The sub units will be in-house made the host computer for data analysis
>> >and
>> >> storage will be a PC (probably with raid HD).
>> >> What communication HW should I use in my sub units?
>> >> What communication HW should I use in the PC?
>> >>
>> >> Regards
>> >> Pini
>> >> 

-- 
Paul Carpenter		| paul@pcserv.demon.co.uk
<http://www.pcserv.demon.co.uk/>        Main Site
<http://www.gnuh8.org.uk/>              GNU H8 & mailing list info.
<http://www.badweb.org.uk/>             For those web sites you hate.

Relevant Pages

  • Re: Data transfer for real time analysis
    ... >The data must be transferred to the PC for online storage and analysis. ... causes some unknown number of actuators to be activated. ... I.E. is there 'n' actuators associated with each sub-unit or does the ... makes control loop time EASIER to achieve. ...
    (microsoft.public.windowsce.embedded)
  • Re: Data transfer for real time analysis
    ... >The data must be transferred to the PC for online storage and analysis. ... causes some unknown number of actuators to be activated. ... I.E. is there 'n' actuators associated with each sub-unit or does the ... makes control loop time EASIER to achieve. ...
    (comp.arch.embedded)