Re: wget options ?

On 28 May 2006 22:43:52 -0700, "Hanselmann" <h_distler@xxxxxxxxxxx> wrote:

I want to rip my own site :-)
So I am under control of almost everything.
What I want to do is a static copy from my site, so in case I need to
shut the db down that I still have at least a static site.

Gruesome, LAMP site?

Any Idea how to do this ?

tar cvzf to another box works for me, but I'm old fashioned
and do me own flat file databases where required ;)

Watch how googlebot crawls your site for an idea how well page
discovery is happening. At worst you may need to write a script
grabbing per page or section with wget. Without specific problem
details, I cannot help. I'm certainly not going to run wget
against a site nominated by anonymous@hotmail on Usenet ;)

Grant.
--
.... The computer scientist, who had listened to all of this said,
"Yes, but where do you think the chaos came from?"
.

Relevant Pages

  • Re: Bash script / wget fans!
    ... if you want wget to run in the background. ... per session, so you don't have to enter it again for that session, or you ... file specified in the script, ... It saves some of the urls to a file without the wget command as ...
    (comp.os.linux.misc)
  • Re: [PHP] Server Stall
    ... One of my scripts are using wget to get external xml data ... Some time $dst host responds very slowly. ... If you are just testing it with another script, ... and then let PHP use the most recent download from wget ...
    (php.general)
  • Re: Bash script / wget fans!
    ... > Very nicely done script. ... > than simply using wget, with the options, etc I'd rather just use wget. ... bash history to it - find this is nicer to use. ... But that's an intrigueing idea - get a file list first. ...
    (comp.os.linux.misc)
  • Re: Perl storing huge data(300MB) in a scalar
    ... I have a very short script that reads the output of wget $URL like so: ... # AT LEAST 8,000 characters ...
    (comp.lang.perl.misc)
  • Re: Looking for a tool to test webserver connectivity
    ... You would think you could use curl or wget to fetch index.html or some such ... Might be able to use grep to ... search output log file for speed number. ... Script would delete the last download, run wget, and if fails mail you ...
    (comp.os.linux)