Re: fortran+java i/o (stupid) problem

From: Larry I Smith (larryXiXsmith_at_verizon.net)
Date: 09/14/04 

Date: Mon, 13 Sep 2004 23:26:01 GMT

simon wrote:
> Thanks Larry,
>
> That keeps my test case from hanging. But I'd like to be able to keep
> the streams open, since the real program repeatedly sends input and gets
> results from the fortran code. It seems like the read thread doesn't
> spring into action until the write thread is closed. Is there a way
> around this?
>
> Thanks again,
> Simon
>
> Larry I Smith wrote:
>
>>

<snip>

The multi-threaded approach is what we always use in unix/linux.

We have several quite large (huge really) Java WEB Services that
spawn C/C++ progs to do compute-intensive work. In all cases we
use the multi-threaded approach. The Java write-thread may live
for several hours, sending GB's of data to the spawned C/C++
process. The Java read-thread lives concurrently, reading MB's
of processed data back from the spawned C/C++ process. When the
Java write-thread has sent the last of the data it closes its write
handle and quits; that causes the spawned C/C++ process to get
an EOF on its next read (or read zero bytes - depending on the
underlying OS); the C/C++ process then closes its read-handle,
does any cleanup, writes the last data back to the Java read-thread,
closes its write handle, and terminates. When the C/C++ process
closes its write handle, the Java read thread will get an EOF on
the next read, close its read handle, and quit.

It is mandatory that each process/thread close its read/write
handles before terminating - otherwise one or the other may hang
awaiting IO.

By default the Java IO pipes are buffered. You may set the IO streams
to unbuffered, but they should get flushed each time you send a newline.
You can always do a manual flush after each write to force the buffered
data down the pipe to spawned process. You also have to pay attention
to the IO buffering in your Fortran program; if it is buffered, the Java
read-thread may not see any output data from the Fortran program
until: a) the Fortran output buffer is full, b) the Fortran program
closes its write handle, c) the Fortran program ends.

In general, when two processes share an IO pipe, the write end
of the pipe should be closed first - thus signalling the reader
to close the read end of the pipe. You have two such pipes in
your sample program: java --> fortran, and fortran --> java

The implementation behind Runtime.exec() varies from OS to OS
and Java VM to Java VM. Some buffer the inter-process pipes,
and some do not. You may have to experiment a little to
figure out what works best for you. I am not familiar with
the underlying code used in the Fortran IO, buffered or
unbeffered - I do not know.

Regards,
Larry 

-- 
Anti-spam address, change each 'X' to '.' to reply directly.

Relevant Pages

  • Re: fortran+java i/o (stupid) problem
    ... The Java read-thread lives concurrently, ... > data down the pipe to spawned process. ... > to the IO buffering in your Fortran program; if it is buffered, ... Some buffer the inter-process pipes, ...
    (comp.os.linux.development.apps)
  • Re: Java and buffer overflows
    ... It is possible that JVM protects against the majority of buffer overflows, ... Java source below. ...
    (Vuln-Dev)
  • Re: progress indicator window with Stop button
    ... The calling code has to pass gets a buffer to write to, but gets will continue writing to that buffer until it reaches a newline. ... Java 1.5's major classfile version is 49. ... restarted, double-clicked the jar file, and... ... the 1.5 or 1.6 compiler to produce class files in the 1.4 classfile format, but that doesn't guarantee that the compiled code doesn't call methods that didn't exist - for that, you need to compile against a 1.4 runtime library, which the compiler -source and -target arguments don't do. ...
    (comp.lang.java.programmer)
  • Re: Passing large C buffers to Java (via JNI) without copying?
    ... C-side buffer in a Java object which will access it directly. ... But remember that /every/ access to that data from Java will then cost more than a direct lookup in a bytearray would, so you should carefully consider whether that overhead will overall be greater than that of a single copy. ... Another factor to consider is that if the JVM allocates the byte buffer then it'll zero the memory for you, which will add a cost not so far off the cost of the copy -- but that might be acceptable if the C code would otherwise have to zero the memory itself. ...
    (comp.lang.java.programmer)
  • Re: How to compare the speed between a Java program and its original Fortran version program?
    ... I have translated an old Fortran program into Java program, ... So it is unfair to include I/O portion, when compare the two ... which demonstrate that the Java app is faster than the Fortran app (in ...
    (comp.lang.java.programmer)