Re: Fedora Core 2 Update: kernel-2.6.6-1.435

From: Ed K. (ed_at_hp.uab.edu)
Date: 06/18/04

  • Next message: Jonathan Rawle: "Re: Making screenshots" 
    Date: Fri, 18 Jun 2004 11:57:43 -0400 (EDT)
To: For users of Fedora Core releases <fedora-list@redhat.com>

    On Fri, 18 Jun 2004, Rodolfo J. Paiz wrote:
    > At 07:32 6/18/2004, Ed K. wrote:
    >> when transferring many small files, http with persistent connections will
    >> blow away an ftp mirror, any day any night.
    >>
    >> What is why the days are numbered. for small files, use http. for large
    >> files (iso), use bittorrent. Plus the bonus of http headers are very
    >> important for caching hierarchies.
    >>
    >> {corrections from anyone......?}
    >
    > Just the comment that you seem to be focusing strongly on the typical use of
    > a single user downloading something large and of high demand, like an iso. As
    > an example, I have a small company which receives a single upload from each
    > one of 40 customers every month, since they are sending us graphics to be
    > published in a magazine. The uploads are usually 3 to 10 files, each ranging
    > from 20 to 100MB, so on average we get 600MB from each customer.
    >
    > Today we use FTP based on the perception I had that FTP is significantly
    > better at transferring large files and maximizing bandwidth. Using max b/w is
    > also important since most of the customers (by virtue of Murphy's law) wait
    > until the very last minute before sending. :-) So HTTP is less efficient you
    > say, and BitTorrent is not applicable.
    >
    > I'm left again with FTP, aren't I? Or scp, but honestly I don't even know
    > which mechanism scp uses.
    >

    scp uses TCP, which would give it the same bandwidth arbitration that http
    uses. If you experiment, I would expect that the deviation from average
    between FTP/SCP/HTTP would be in the noise.

    You have just pointed out that generalities _always_ fail for some case.
    HTTP is impractical for uploading files larger then 4MB, it requires additional
    configuration of the servers, and caches that should be in the way.

    As you know, ftp is insecure, but you may have secured the files some other way.
    I would suggest using pure-ftpd, it has great anonymous upload features. In
    fact, I use ftp upload too, only because the client requires it.

    So, to summarize, FTP is not significantly better at transferring large files
    and maximizing bandwidth. It is just easy to use and established. I don't know
    if FTP supports continuing a disconnected upload attempt. (In other words I
    haven't seen it in all my readings) There might be simple custom solution
    that would help in the event of a disconnection that would prove to be more
    beneficial than using FTP.

    Just to give you some other methods, on my current project, I have to transfer
    about 6GB of files, each one no larger then 100K from one computer to the next.
    FTP/SCP/HTTP are all slow. Using tar and mbuffer, I can transfer the files
    sustaining 11MB/s. That is the maximum the 100MB switch will move the packets.
    9 minutes 18 seconds.

    Different problems, different solutions.

    ed

    Security on the internet is impossible without strong, open,
    and unhindered encryption. 

    -- 
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: http://www.redhat.com/mailman/listinfo/fedora-list
  • Next message: Jonathan Rawle: "Re: Making screenshots"

    Relevant Pages

    • Re: http uploading
      ... It's not unusual for ftp password to be different. ... | It seems that the password that you use to upload through http is the FPSE ... |> My problem with http has only appeared now i have crossed over to Vista. ...
      (microsoft.public.publisher.webdesign)
    • Re: Problem with FTP
      ... Both protocols will upload your site files, but http uses FP extensions on ... the server (means your server host had to install them), whereas ftp is the ...
      (microsoft.public.publisher.webdesign)
    • Re: Word = Web
      ... you may use standard methods of FTP ... or HTTP to upload FP files. ... Here are some beginner links. ...
      (microsoft.public.word.docmanagement)
    • Re: Upload file without user interaction - VBA
      ... available method is using an HTTP form and the POST method. ... You probably need to use FTP, not HTTP. ... Your posting first says I want to upload files which is FTP. ...
      (microsoft.public.excel.programming)
    • Re: Fedora Core 2 Update: kernel-2.6.6-1.435
      ... for small files, use http. ... Today we use FTP based on the perception I had that FTP is significantly ... better at transferring large files and maximizing bandwidth. ... is also important since most of the customers ...
      (Fedora)
    Loading