Re: A script for scanning the internet for one particular index.html?



Hi Allen and the group -

On Sat, 22 Sep 2007 12:11:23 -0500, Allen Kistler
<ackistler@xxxxxxxxx> wrote:

Floyd L. Davidson wrote:
Man-wai Chang ToDie <toylet.toylet@xxxxxxxxx> wrote:
I want to know how I make a script for scanning the entire internet
(127.0.0.1 - 255.255.255.255)
for one unique index.html file (I got the index.html file I'm looking for)

use wget to fetch the index.html into a file then grep -i its content.

How many decades will it take a PC to search half the Internet that way?

Not quite half.

Routable addresses are (theoretically):
the range 1.0.0.0/8 - 223.0.0.0/8
minus:
10.0.0.0/8
127.0.0.0/8
169.254.0.0/16
172.16.0.0/12
192.0.2.0/24
192.168.0.0/16

In the meantime, however, this request seems an awful lot like what bots
do to look for new servers they can infect.

That comment brings up another point. With the proliferation of
virtual named hosts, going by IP address is going to miss a lot. I
have multiple domains on my server, and in some cases multiple hosts
per domain. And the default container (which is what access by IP
address without a host name or with the IP address as host name would
get) does not contain any of those sites.

--
Ken
http://www.kensims.net/
.



Relevant Pages

  • Re: The libntp resumee...
    ... you have too much faith in ntp. ... IF he is using his own servers (not outside ... using seven DIFFERENT poll intervals, one for each server because seven ... Currently we observe that both entry hosts can both become restricted due to ...
    (comp.protocols.time.ntp)
  • Re: Slow DNS Lookup - Debian
    ... >> hosts: files nisplus nis dns ... >> and if you don't run NIS or NIS+, you can speed things up slightly by ... use tcpdump to see that you are using responsive name servers. ...
    (comp.os.linux.networking)
  • Re: IPTables rules and hosts that use DHCP
    ... authentication requests with all sorts of passwords and/or usernames. ... the rules blacklist hosts that attempt to connect more than ... my sister and I act as backup servers for each ... Do you HAVE to have your SSH server on port 22, ...
    (comp.os.linux.misc)
  • Re: Is there a time module to help with this...
    ... I'm just a beginner in Perl, but I use cron and use Net::Ping for the hosts. ... I have something similar using perl's DBI to test oracle: ... So let's say I want to ping each server every 5 ... I don't want to code a loop where I ping all the servers and then ...
    (perl.beginners)
  • Re: IPTables rules and hosts that use DHCP
    ... password authentication requests with all sorts of passwords and/or ... the rules blacklist hosts that attempt to ... my sister and I act as backup servers for each ... Do you HAVE to have your SSH server on port 22, ...
    (comp.os.linux.misc)