Re: wget - the question for advanced users

There may be a link in one of the directories to the main page
of the site.

Yes, such links exists but I want to force wget not to follow them.

I used -R option but it doesn't work as I expected. The documents from the
main directories are deleted but wget follows the links in them. The problem
is that the documents in the main directory have so many links that I can't
and I don't want download them (I would download the contents of all server
which means terribly, terribly huge amounts of data).

I tried to use -I option. In such situation wget downloadts the html pages I
want but it skips the photos placed on other servers. :((

I tried also the option -X. In such situation the photos are downloaded but
dependning on what I wrote after -X wget starts to download the contents of
all server or wget downloads only one html document.

I have impression that wget is not able to fulfill the task I want to
realise but I wrote my problem here hoping that maybe someone will find a
nice trick.


---------------------------------------------------

http://www.pbase.com/piotrstankiewicz


.

Relevant Pages

  • RE: (OT) Bit Torrent usage ...
    ... if i have a server setup, and allow you to access the file, you can use ... wget, and you're pretty much going to scream at the slow speed of my upload ... bittorrent really won't do you much good.... ... you're using to download the file. ...
    (Fedora)
  • Re: How to see http*.mp3 size before download ?
    ... mallicious attack from the server. ... Wget is non-interactive, meaning that it can work in the background, ... Wget can follow links in HTML and XHTML pages and create local ... it will instruct the server to continue the download from ...
    (comp.os.linux.networking)
  • Re: (OT) Bit Torrent usage ...
    ... > If you use a conventional tool, even wget, you are making one connection to ... If that server goes down, or slows down, your transfer ... > More than once, I've lost a complete Red Hat download because, after ... If a single connection fails, you only lose a portion of the data ...
    (Fedora)
  • Re: Free Metalworking Plans
    ... NRA LOH & Endowment Member, Golden Eagle, Patriot"s Medal. ... | I Dl'd the lathe plans, ... Use something like Wget ... or some other worthy download manager to retrieve it. ...
    (rec.crafts.metalworking)
  • Wget usage : request for comments
    ... I am going to start a small project to analyze ... 8 websites with hyperlinks, images, js, etc.. ... I will use wget as a crawler (I like command ... seems I can't download the results in text format. ...
    (comp.os.linux.misc)