Re: wget, how do I download when links are to a php ...?


Tony Lawrence wrote:
jameshanley39@xxxxxxxxxxx wrote:
Many thanks for all the different solutions. I think i'll try a few of
them, for the sake of improving my linux skills

I appreciate the wisdom of one of the responses about the unix
philosophy.. My question had the un unix like presumption that one
command would do the job !

in practice, i've found that sometimes, one command can do a job but it
uses some ugly and quirky switches / options. And it'd have been better
to use 2 commands
In one case, there were more switches, to fix th quirks. As one linux
guru once said to me "more buttons indicate that something/some aspect
of it, is broken" I think , at the time, he was referring to how
globbing is handled by the bash. He said it's enough of a solution to
work from interactively with the shell, but not predictable enough to
use in a shell script. The 100% / predictable solution may use more
keystrokes. But interactively, the 90% solution can work fine most of
the time as long as you know its limitations. This real unix philosophy
is not discussed much(or is it?) and doesn't seem so neat.

I'm not sure what you think the "real Unix philosphy" is so I don't
know why you think it's "not so neat". There's a book I didn't like (
http://aplawrence.com/Books/linuxunixphilosophy.html ) that covers that
subject very completely; maybe you'd like to pick it up.


I did read it, it spoke a lot of small programs that do the job they
specify. nothing more, nothing less. And stringing the small programs
together with pipes.

My point is that from the little I know of linux, the reality is more
complicated. e.g. there are commands that do one thing well and another
thing in a messy way.

There are quick 90% solutions and long 100% solutions in places where
there could easily be a short 100% solution. I'll give an example of
this at response to the next paragraph, with a BUT. note- Within that
respone to ten et paragraph, the example is there, about displaying
only files [file that aren't directories] with ls.



There is a difference between ad hoc solutions and portable solutions.
For example, consider a case where I already know that there is one
directory underneath where I sit now and that it contains nothing but
text files. Additionally I know that the only time "foo" appears in
any of those files is in the word "foobar".

If I need to find "foobar" in those files, "grep foo */*" is an ad hoc
solution that does exactly what I want. However, it would be a lousy
answer to someone asking the general question "How do I find 'foobar'
in my files?".


my current situation is not ideal for responding well. I haven't had
much experience in unix and don't have access to it at the moment.
When I get free time I sometimes run to it and then run away from it
again..
I'll run to it again when I get some free time to play with it, since
i've rethought some of te philosophy..

I can't really comment on the grep foo */* because I can't try it out
at the moment. I know you intended to give a simple example that I
wouldn't have to try out.. And I know what it does. But i'd need to try
it with variations to know it properly enough to discuss this..

When I wrote
"There are quick 90% solutions and long 100% solutions in places where
there could easily be a short 100% solution"

I'm largely familiar with DOS, which I used frequently over a decade
ago.. DOS let me type dir /ad (which displays directories). Or, dir
/a-d (which displays non directories i.e. files). And it allowed dir
/ah <-- to display hidden files, and by chance, I found I could even
do things like dir /ah-d(hidden files not directories).

The first command I learnt in linux was ls . My experience with unix
doesn't go far beyond that, but just looking at that and little else, I
found that it had 90% solutions, and things it didn't do well.. I
discovered that just trying to get it to do the simple things that I
mentioned that DIR does..

ls on its own can't display ' only files '. First, it has no switch
for that. Secondly, I tried some ways to get it to do so, and don't
know why they didn't work..

I'm writing these from memory

the thought process to get ls to display " just files "
you see where I hit a wall..
ls -d <-- displays directories and doesn't expand them.. fine

ls -d */ <--- works out the same

ls *[^/] <- why doesn't it work ? it's in the bash shell,

I know ls *[^/] is not even a portable solution either 'cos it's only
for the bash..

If I look at DIR in DOS, it does what it does. A simple command. It
doesn't do anything else, and it does what it does, completely.
I'm suprised that ls doesn't include those simple functions..
I'm not saying that DOS is better than linux. I'm saying that whatever
the unix philosophy is, it's not like the book made out.. And in the
sense of commands that just serve their purpose, DOS's DIR really lives
up to that, but linux's "ls" doesn't.


Perhaps I should discuss this another time when I'm back trying linux .
that'll be when I get some free time. Hopefully in a few months, then
i'd be more useful in discussion


As that poster in this thread wrote.. linux is all about stringing
commands together. I hoped i'd be able to do something with ls alone,
just as DIR could, cos it was such a simple thing. But it requires ls
| grep . Fair enough , linux is linux , i'll have another look at it,
this time not expecting commands to contain the complete functions of
their corresponding DOS ones.

Regarding tihs wget question... I should have guesesd that I should've
asked the question in terms of will wget fit into the solution and
how.. Rather than how do I do it in wget.. In the windows philosophy,
somebody would've produced a better version of wget that'd have done
it.. DOS and the DOS Prompt allows quite good redirecting output with
commands, though i'm interested in linux. I'm suprised that people
haven't done both..

Produced a better version of the program AND of course as you and
others have done here, strung commands together to do it..

Why isn't there a new version of ls that is backwards compatible, that
lets you display "only files [that aren't directories]" ?

it's such a basic function

why hasn't the said functionality regarding wget, been added to wget?

Stringing commands together is nice, but commands should be complete
too

I hope to post abotu this when i've looked at *nix again, and have some
more time to play with it.

the fact that *nix has no term for files that aren't directories, is no
excuse for ls not having that functionality, it just makes it even
stranger.. And anyhow, i'd be interested ina good reason of why ls
*[^/] didn't work in the bash!
Though I may only be able to research the given reason in a few months
time

.