Re: scripting fun

Bart Silverstrim wrote:
Okay...closer...combining what others have given (thank you everyone!),

Not wanting to start a holy war but shell scripting and "fun" aren't
normally words I join without some form of negative modifier in there. As
you've run across issues with parameter passing and globbing just make the
entire exercise a thousand times harder than it needs to be.

Python, Ruby, Perl, all better suited to this task simply because you
don't have to worry about the shell doing something wicked with your data as
it streams hither and yon.

Just as an example here's a sloppy, off-the-cuff version of your script in
Python:

import os
log = '/var/log/apache-perl/access.log'
ipbin = '/sbin/iptables -L INPUT -v -n'
iptables = '/tmp/iptables'
newips = {}
oldips = {}

lines = open(log, 'r')
for line in lines:
if line.count('slurp') > 0:
newips[line.slpit()[0]] = True
lines.close()
os.system('%s -L INPUT -v -n > %s')
lines = open(iptables)
os.unlink(iptables)
for line in lines:
if line.count('source') > 0:
continue
try:
oldips[line.split()[7]] = True
except IndexError:
pass
iptables.close()
for ip in newips:
if ip not in oldips:
os.system(%s -A INPUT -s %s -j DROP' % (ipbin, ip))


Yes, I admit it is a tad longer (and probably does have a lurking bug or
two since I did not run it) but what prompted me to write this message is
three fold:

1:
for line in lines:
if line.count('slurp') > 0:
newips[line.slpit()[0]] = True

vs.

grep -i slurp /var/log/apache-perl/access.log |awk '{print$1}' >
~/temp/tmp.txt
sort ~/temp/tmp.txt > ~/temp/tmp2.txt
uniq ~/temp/tmp2.txt > ~/temp/slurps.txt
-----
This uses a dict as a uniq filter. No need to worry about uniq vs sort -u
vs sort-un. The key is the IP and setting same IP to True 80 times still
results in exactly 1 entry.


2:
for ip in newips:
if ip not in oldips:

vs.

join -v2 < (sort <~/temp/blocked.txt | uniq ) < (sort <~/temp/slurps.txt
| uniq ) > ~/temp/newaddresses.txt
-----

Look, I've been poking around at Linux and other Unix(-like) system for
almost 2 decades now. Even so I look at that shell line and want to just bang
my head against the desk. As you found out join seems to have some issues
with sort order. Besides, shouldn't they already be sorted and uniqed from
previous runs?

Regardless, look at the Python lines. Dicts are hashes. That means they
are inherently unsorted. It's obvious I don't care about sort order. Also
since I used the same trick of generating each key once I just iterate over
the newips (from the log file) and check to see if they are in the oldips
(from iptables). Probably not the fastest way to do it in Python but, hey,
its rare I need to code for speed on utility scripts whereas readability and
quick, first off, runs count for a lot. ;)

The third reason is simple. I saw your message and thought, "Wow, neat,
that'd be fun to write in Python." Seriously. I enjoy the mental exercise of
doodling with little tools like this in Python. It's fun scripting versus, as
I read it, the ironic scripting 'fun'. ;)

--
Steve C. Lamb | But who decides what they dream?
PGP Key: 1FC01004 | And dream I do...
-------------------------------+---------------------------------------------

Attachment: signature.asc
Description: OpenPGP digital signature

--
ubuntu-users mailing list
ubuntu-users@xxxxxxxxxxxxxxxx
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Relevant Pages

  • Re: Batch commands on Windows
    ... Each of your system calls spawns it's own separate shell with its own ... >> would work in Python? ... > then delete the batch file, but that seems like an awful hack. ... Shell scripting started ...
    (comp.lang.python)
  • Re: Batch commands on Windows
    ... Each of your system calls spawns it's own separate shell with its own ... >> would work in Python? ... > then delete the batch file, but that seems like an awful hack. ... Shell scripting started ...
    (comp.lang.python)
  • Re: scripting fun
    ... doodling with little tools like this in Python. ... It's fun scripting versus, as ... I read it, the ironic scripting 'fun'. ... To be honest, I would love to get more into programming, I simply ...
    (Ubuntu)
  • Re: Batch commands on Windows
    ... >> the other niceties of python. ... And that seems like the most basic thing that a scripting ... although MS-DOS shell scripting is certainly a small subset of scripting ... This confused me when trying python scripts in MS Windows after using ...
    (comp.lang.python)
  • Re: Copy database with python..
    ... I get the distinct impression that you're trying to replace simple shell ... scripting with Python. ...
    (comp.lang.python)