Re: Comments on the fastestmirror plugin

Patrick O'Callaghan wrote:
On Sat, 2010-03-13 at 16:52 -0500, Bill Davidsen wrote:
Patrick O'Callaghan wrote:
The yum fastestmirror plugin (yum-plugin-fastestmirror) claims to
evaluate the speed of a bunch of repo mirrors and use the fastest one
relative to the user's location.

However AFAIK what it *actually* does is make a test connection to the
to the candidate mirrors and order them according to response time,
which in many cases is dominated by network latency, which can distort
the results. For well-connected user machines in first-world countries
it probably doesn't matter much, and may have the beneficial effect of
spreading the load over a wider range of mirrors, but for those of us in
a less privileged position it can matter a lot. Ironically, these are
the cases where such an optimization could do the most good.

And there you have the heart of the problem, the evaluation is not remotely
correct for most cases. It would be worth adding code to download some small RPM
from a number of sites and measure b/w for something real. However, disabling
the feature works, too.

Sadly, downloading a "small" RPM is unlikely to give very reliable
results either. Due to TCP slow-start, a stable effective b/w may only
be reached after some 10's of kb have been downloaded.

Wow, I wasn't talking dialup. Tens of kb is a few ms on anything useful for
major D/L, the slowest DSL I've ever seen sold was 768kb, or max of ~80kB/s.
Figure 100kB per site and stop after the ten fastest are within 5-10% of each
other (that's probably max line speed). Base the speed calc on the last 10k of
100kB to get the steady state value. You don't really care if the process is
slow, once the table is built you use it. Actually, rechecking the five fastest
is probably practical.

This is not an easy problem to solve.

Fortunately, you don't have to find the best, virtually all of the top sites
will be close unless they're overloaded. The perfect is the enemy of the good
enough.

--
Bill Davidsen <davidsen@xxxxxxx>
"We have more to fear from the bungling of the incompetent than from
the machinations of the wicked." - from Slashdot
--
users mailing list
users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Relevant Pages

  • Re: Comments on the fastestmirror plugin
    ... evaluate the speed of a bunch of repo mirrors and use the fastest one ... It would be worth adding code to download some small RPM ... a stable effective b/w may only ... major D/L, the slowest DSL I've ever seen sold was 768kb, or max of ~80kB/s. ...
    (Fedora)
  • Re: capping the apt-getting of packages from ftp.xy.debian.org to < 50KBps?
    ... archive mirrors. ... capping the download speed on its servers by this much then I would ... I gather that you've tried several different mirrors, ... I only have a couple of computers running debian so running a local ...
    (Debian-User)
  • Re: Comments on the fastestmirror plugin
    ... evaluate the speed of a bunch of repo mirrors and use the fastest one ... It would be worth adding code to download some small RPM ... a stable effective b/w may only ...
    (Fedora)
  • [Full-disclosure] Re: Download Accelerator Plus can be tricked to download malicious file
    ... DAP searches for all its mirrors from mirrorsearch.speedbit.com ... I have no knowledge about HOW the mirrors are gathered. ... > one of the popular and best download manager for windows. ... > providing the file is DOWN or due to network congestions the users may ...
    (Full-Disclosure)
  • Re: Yum annoyance: "[Errno -1] Metadata file does not match checksum" over and over
    ... It's common for this to happen around a dozen times or more, which since the file is 3.8M can add up to 50M just to download metadata to find out whether there are updates. ... So it uses the live mirrorlist {by default - and customized to your country -if more than two in country mirrors seem OK, otherwise the general all mirrorlist}, connects at random to one of the mirrors, and downloads the apparently ~out of date~ metadata file. ... Modify the $basearch to be your fedora architecture ...
    (Fedora)