Re: Wget hangs: process never terminates

On Tue, 25 Apr 2006 11:01:15 -0700, ponga wrote:

Hello all.

I use wget in a a shell script to do some offsite backup stuff, so the
script is kicked off using cron. However, ever time - wget hangs in the
middle of downloading a file! The result is, I get a partially downloaded
file and the cron job NEVER terminates. I ran just the wget command from a
shell with degug output to see what I could see, but nothing of interest
is shown! It shows the dl progress, then sometime in the middle
(arbitrary) - wget just hangs... the estimated time goes to infinity and
file never completes and wget does not terminate!! After I Ctrl-C, then
re-run wget with the -c option, the dl picks up from where it left off and
the file is eventually downloaded.

Specifics: File is 800MB in size coming over a 1.5Mb connection using
http.

Can someone please offer advice as how to fix this?? I would have thought
that wget would be more robust and I would try curl, but I've seen curl do
this same thing before too (with urpmi). My other thought was to have
ANOTHER cron job run that finds the hung wget job, terminates it, then
re-runs wget with the -c option... but thats kinda messy.
Can anyone suggest anything??

TIA, any help would be GREATLY appreciated!!

-Mike


Hello

1 question and 2 comments

Q1 : If you try a manual run of your backup script does it also goes on
forever ?



C1 : If you try to use "curl" instead of "wget" is the same thing
happening ?

C2 : IPtraf could help you determine the status of the TCP connection :
Available at :
http://iptraf.seul.org/
http://freshmeat.net/screenshots/4549/4549/

But thing as simple as :

watch -d netstat -a -n -t

in a running terminal or console might be enough...



Hope it help...
.

Relevant Pages

  • Re: Bash script / wget fans!
    ... if you want wget to run in the background. ... per session, so you don't have to enter it again for that session, or you ... file specified in the script, ... It saves some of the urls to a file without the wget command as ...
    (comp.os.linux.misc)
  • Re: kill process and child
    ... Script A is supposed to start/stop some other script, ... and B is still running with wget as its child. ... and the main process A.sh terminates, then the process will be controlled ...
    (comp.unix.shell)
  • Re: [PHP] Server Stall
    ... One of my scripts are using wget to get external xml data ... Some time $dst host responds very slowly. ... If you are just testing it with another script, ... and then let PHP use the most recent download from wget ...
    (php.general)
  • Re: [SLE] Setting a script to automatically download a file from nai.com ftp site?
    ... > Good day Damian, ... > cron can be used to repeat execution of kommands at certain intervals. ... > wget can fetch files from ftp sites. ... Put this in a script and then run it from cron. ...
    (SuSE)
  • Re: shell script
    ... vorrei fare un piccolo script per andarmi a prendere una serie di file ... Qualcuno sa correggermi i molti errori, non essendo io per nulla esperto di ... wget http://hereis.my.location/data"$take".dat ... Se il file con quel contenuto si chiama "script", ...
    (it.comp.macintosh)