[SLE] wget bug or dumb user?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi,

I'm trying to download the whole 9.3 update tree for a friend without
internet access. I'm doing it this way (mind: some very long lines):

#!/bin/bash
wget --continue --timestamping --quota=1800m -r -l3 --limit-rate=90k \
--no-remove-listing -nH --cut-dirs=5 --output-file=LOGFILE \
--reject "OpenOffice*-af-*rpm,OpenOffice*-ar-*rpm,OpenOffice*-be-*rpm,OpenOffice*-bg-*rpm,OpenOffice*-ca-*rpm,OpenOffice*-cs-*rpm,OpenOffice*-cy-*rpm,OpenOffice*-da-*rpm,OpenOffice*-de-*rpm,OpenOffice*-el-*rpm,OpenOffice*-et-*rpm,OpenOffice*-fi-*rpm,OpenOffice*-gu-*rpm,OpenOffice*-hr-*rpm,OpenOffice*-hu-*rpm,OpenOffice*-it-*rpm,OpenOffice*-ja-*rpm,OpenOffice*-ko-*rpm,OpenOffice*-nb-*rpm,OpenOffice*-nl-*rpm,OpenOffice*-nn-*rpm,OpenOffice*-pa-*rpm,OpenOffice*-pl-*rpm,OpenOffice*-pt-*rpm,OpenOffice*-ru-*rpm,OpenOffice*-sk-*rpm,OpenOffice*-sl-*rpm,OpenOffice*-sv-*rpm,OpenOffice*-xh-*rpm,OpenOffice*-tr-*rpm,OpenOffice*-zh-*rpm,OpenOffice*-zu-*rpm" \
--exclude-directories=/pub/linux/suse/ftp.suse.com/suse/i386/update/9.3/rpm/x86_64 \
"ftp://ftp.gwdg.de/pub/linux/suse/ftp.suse.com/suse/i386/update/9.3/";


It is the "--exclude-directories" option that is causing me the biggest
problem: after I added it, it never stops, it goes round and round
downloading (skipping, actually) the same files. I had to make it log to a
file to make sure, it was flipping by so fast.

I track the download marking the ".listing" entries in the log (trimmed
for clarity):

- --20:30:58-- ftp://ftp.gwdg.de/pub/linux/suse/ftp.suse.com/suse/i386/update/9.3/
=> `i386/update/9.3/.listing'
20:31:01 (14.83 MB/s) - `i386/update/9.3/.listing' saved [5412]
20:31:06 (71.81 KB/s) - `i386/update/9.3/deltas/.listing' saved [2574208]
20:31:16 (27.28 KB/s) - `i386/update/9.3/patches/.listing' saved [241797]
20:31:24 (27.75 KB/s) - `i386/update/9.3/patches.obsolete/.listing' saved [121069]
20:31:26 (18.86 MB/s) - `i386/update/9.3/rpm/.listing' saved [5655]
20:31:30 (69.83 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [6910062]
20:32:02 (46.33 MB/s) - `i386/update/9.3/rpm/i686/.listing' saved [34980]
20:32:04 (22.74 KB/s) - `i386/update/9.3/rpm/noarch/.listing' saved [426240]
20:32:07 (50.95 KB/s) - `i386/update/9.3/rpm/src/.listing' saved [2124577]
...
Not descending to `pub/linux/suse/ftp.suse.com/suse/i386/update/9.3/rpm/x86_64' as it is excluded/not-included.
...
20:32:13 (70.47 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [7107435]
20:32:38 (79.04 KB/s) - `i386/update/9.3/rpm/i686/.listing' saved [36146]

Notice that it has gone round again to download the same
i386/update/9.3/rpm/i586/ directory again!

And it continued...

...
20:32:48 (72.20 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [7304808]
20:33:25 (72.53 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [7502181]
20:33:57 (70.83 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [7699554]
20:34:33 (71.51 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [7896927]
20:35:14 (70.25 KB/s) - `i386/update/9.3/rpm/i586/.listing' saved [8094300]


Before I killed it, it had gone round 7 times in 5 minutes - fortunately,
it only really downloads the ".listing" files, due to the "--continue"
switch.

What is happening? :-O

[...]

Those ".listing" files were increasing in size... yeap, the server was
syncing. A while later retried, downloaded some files related to ethereal,
and finished normally. Then I tried some hours later, and it also worked
normally.

Weird...

The problem is solved, but... why the fact that the server was adding some
new files caused wget to get into a never ending loop?


- --
Cheers,
Carlos Robinson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Made with pgp4pine 1.76

iD8DBQFEBMfntTMYHG2NR9URAinRAJ9KoVoidmQXcmAe53w77lS08bHvOACgj2LF
tCluItIMup07DXQ3gY2gb0c=
=5pZi
-----END PGP SIGNATURE-----


--
Check the headers for your unsubscription address
For additional commands send e-mail to suse-linux-e-help@xxxxxxxx
Also check the archives at http://lists.suse.com
Please read the FAQs: suse-linux-e-faq@xxxxxxxx

Relevant Pages