data written to file missed by reader

Hello All,

We currently run processes that communicate by writing an encoded
message to a flat file while other processes are reading the file. A
message that was written to the file by one process (fsync is not
called after each write) was missed by the reader. A second message
was written 1 ms later. When I say "missed", I mean that the process
read the second message and continued processing. Another process
reading messages from the same file came through ~200 ms later and read
the missed message and then the second message in the expected
sequence. Is there some kind of synchronization issue after the data
has been written that could have caused the reading process to miss the
data? Perhaps during the flush to disk? Is there any inter-process
file access management performed in the virtual layer?

We are running on linux kernel 2.4.21-32.0.1 using multiple processors
(smp) and SCSI disks in a RAID 1 configuration.

Thanks,
--Sean

.

Relevant Pages

  • Re: data written to file missed by reader
    ... message to a flat file while other processes are reading the file. ... called after each write) was missed by the reader. ... read the second message and continued processing. ... My bet is that there is a flaw in your scheme for detecting new ...
    (comp.os.linux.development.system)
  • Re: DB -> Flat file, did I waste my time?
    ... An array typically gets created to ... The question you asked, "Flat file, did I waste my time?" ... reading a simple table. ...
    (comp.lang.php)
  • Re: File I/O vs. MySQL
    ... the server from your http request, as your connection may timeout regularly. ... > a performance difference between reading a flat file vs. reading a ...
    (comp.lang.php)
  • Re: Modifying the input
    ... > So how do I change records in the file I am reading from? ... Write to a new, temporary file, while reading through the original file. ... perldoc -f rename ... > It HAS TO BE A FLAT FILE. ...
    (comp.lang.perl.misc)
  • Re: What is this command doing?
    ... It's supposed to be reading in a flat file and ... filtering out the duplicates based on a key in the file... ...
    (comp.unix.shell)