Re: wget - the question for advanced users

On Fri, 29 Dec 2006 00:28:01 +0100, Piotr wrote this:

Did you try the --mirror option?

Start over use the --mirror option for your homepage. Enable log file
then maybe you can see what Wget is doing.

It doesn't work as I would expect. Even if I use -m option wget starts to
download documents from the main directory / and I don't know why. :((

Ex. It downloads www.pbase.com/login.html

Even if I reject that page with -R option, wget follows the links from the
login.html page. :(


---------------------------------------------------

http://www.pbase.com/piotrstankiewicz

Hmmm sorry I'm not following you too well.
You want to follow the links for sub directory html and photos but exclude
parent directories? If you've run wget and have every thing then you
can delete what you don't need right ?

Following the links should download pages linked to your html even when
they link to parent domains such as the login.html I think?

Anyway I think you should try the --mirror with --exclude to obtain
the website clone that you want.

Also, a link on wget for following links if you haven't read it yet:

http://www.gnu.org/software/wget/manual/wget.html#Following-Links


.

Relevant Pages

  • Re: Free Metalworking Plans
    ... NRA LOH & Endowment Member, Golden Eagle, Patriot"s Medal. ... | I Dl'd the lathe plans, ... Use something like Wget ... or some other worthy download manager to retrieve it. ...
    (rec.crafts.metalworking)
  • Wget usage : request for comments
    ... I am going to start a small project to analyze ... 8 websites with hyperlinks, images, js, etc.. ... I will use wget as a crawler (I like command ... seems I can't download the results in text format. ...
    (comp.os.linux.misc)
  • Re: [unix, console] : something like wget but with forms + authentification
    ... >> download the response after submitting your name and password, ... > tools, like lynx, links, elinks, wget, don't seem to access to this ... I used wget to download the pages you linked to, ... html expert, but perhaps I have been exposed to a little more ...
    (comp.unix.shell)
  • Re: JScript question
    ... GNU Wget is a free utility for non-interactive download of files from the Web. ... It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. ... Thus Wget can see if the remote file has changed since last retrieval, and automatically retrieve the new version if it has. ... it will instruct the server to continue the download from where it left off. ...
    (microsoft.public.scripting.jscript)
  • Re: Please Help W/Wget.
    ... Me and wget are *old* friends... ... I wanted to download an area on geocities, ... You can specify how long you should wait between each download. ... init-file -- wget will put everything in your ...
    (comp.os.linux.misc)