Re: Network file transmission with speed of raw disk read. How can that be?!?

From: Jens Kramel (no_at_spam.org)
Date: 07/08/05 

Date: Fri, 8 Jul 2005 11:43:28 +0200

"David Schwartz" <davids@webmaster.com> wrote in message
news:dalfcm$dpb$1@nntp.webmaster.com...
> It would report the error when you tried to close the connection down.
>

How could it do that? Are we talking about the simple close(..)-call or do
you think of any other way to shut the connection down?

> If the disk is faster than the network, the network buffer is always
> full. If the network is faster than the disk, the network buffer is almost
> always empty. Either way, you will get parallelism.
>
>
> DS

So, let's build an example from this (used values might be a bit unrealistic
;)
Let's assume that disk is significantly faster than network and that the
whole network buffer is already full.
The program loops on:
read 100 bytes into buffer x
send the 100 bytes in buffer x

In this case, we would nearly lose the advantage of parallelism, right? The
send(..)-call has to wait until there are 100 bytes freed in the buffer,
which means it has to wait until the network really tranferred 100 bytes of
data. until this is completed, the send(..)-call can't return and the next
read(..)-call won't happen. And because the disk is always faster than the
network in our example, the disk will read the next 100 bytes again before
the network freed up another 100 bytes, so there will be another significant
delay.

Please, correct me if I'm wrong. And thanks again, of course!
Jens

Relevant Pages

  • Re: why does scp not use all my cores on a multi-cpu machine?
    ... The buffer size issue is actually part of the I/O bottleneck. ... In the FAQ for hpn-ssh there is an important couple of Q&As ... is the speed of your network connection to the Internet. ...
    (comp.os.linux.misc)
  • Re: Socket write behaviour is inconsistent?
    ... I enabled network tracing and in all cases the bytes written are the ... copy 1 byte to buffer, copy many bytes to buffer, copy one byte to ... "response" buffer already includes a termination byte. ... The server isnt supposed to close the connection; ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Network file transmission with speed of raw disk read. How can that be?!?
    ... the network connection is completely dead. ... > whole network buffer is already full. ... > In this case, we would nearly lose the advantage of parallelism, right? ... And because the disk is always faster ...
    (comp.os.linux.development.system)
  • Re: Windows buffering downloaded data?
    ... whenever I start to download a file or watch a video etc. from the net it takes really long time before anything starts to happen. ... When I test the network download speed with that computer, first it takes forever before the test starts to download the data and then it’s suddenly over and the test results shows that my network speed is something like 55M. ... It seems like the PC is buffering the data coming in and only then, when the buffer is full (or the connection is closed before the buffer getting full) the PC uses the data. ...
    (microsoft.public.windowsxp.network_web)
  • Re: Windows buffering downloaded data?
    ... whenever I start to download a file or watch a video ... When I test the network download speed with that computer, ... buffer is full (or the connection is closed before the buffer getting full) ...
    (microsoft.public.windowsxp.network_web)