Re: Sudden increase in size of Debian?

From: Alex Malinovich (demonbane_at_the-love-shack.net)
Date: 09/26/03

  • Next message: Jacob Anawalt: "Re: Anti-Spam ideas for usenet/list harvested email addresses" 
    Date: Thu, 25 Sep 2003 19:52:32 -0500
To: debian-user@lists.debian.org

    On Wed, Sep 24, 2003 at 05:33:17PM -0600, Jacob Anawalt wrote:
    >
    > Kevin McKinley said:
    > > On Wed, 17 Sep 2003 00:21:55 -0600
    > > Jacob Anawalt <jacob@cachevalley.com> wrote:
    > >
    > >> Now you have me interested. Do you already have a script to mirror only
    > >> stable and unstable with rsync? I think I would try only mirroring
    > >> stable with such a script, but I thought it would take having a program
    > >> parse things like the Packages files for each release and
    > >> main/contrib/non-free sub folder and <arch> that you were after.
    --snip--
    > Ok, I'm only slightly less lazy in this area today. Here's a half-baked
    > idea of what I thought it would require.
    >
    > 1) Run rsync for just the dists/* files and links that you want, but
    > specifically getting all the Packages files for the targets you want but
    > at least dists/woody/main/binary-i386/Packages* if you're after i386.
    >
    > 2) Call a script that does this - only make yours better ;)
    >
    > Modify the package reading script to read the Packages file for all
    > release, target and arch you are interested in, writing the output into
    > rsync_packages.txt. Use whatever language you like. Here's some Perl in
    > your eye.

    I've had a script that does this very thing for quite a while
    now. I've mentioned it to a few people on here before, but I've never
    officially "released" it so I don't really know if anyone but myself
    uses it. I run it as a nightly cron job to update my sid mirror (which
    is what it was initially written for) and it's been working with no
    problems for almost 2 years now.

    It downloads Packages.gz for sid (it's easy to change this), reads
    through it, and populates an array with the paths and filenames for
    all the included debs. It then searches your current mirror for which
    files need to be updated, and flags the old versions of those files
    for deletion.

    The old files are then deleted (if you run it interactively it'll
    prompt you first) and then starts an rsync for any packages that it
    doesn't have.

    It's written in perl though it calls a few system commands (namely
    'find' and 'rsync'). If you're interested, you can get it at:

    http://www.the-love-shack.net/geek.html

    The actual script is sidmirror.pl, but you'll also need to get the
    other related files to handle cleaning up old files, etc.

    It's not extremely well documented, but the variables at the top of
    file are pretty clear so you can set it up to do more or less what you
    want. If you need any help with it, or if you decide to use it, please
    do let me know. :) 

    -- 
Alex Malinovich
Support Free Software, delete your Windows partition TODAY!
Encrypted mail preferred. You can get my public key from any of the
pgp.net keyservers. Key ID: A6D24837



    -- 
To UNSUBSCRIBE, email to debian-user-request@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
  • Next message: Jacob Anawalt: "Re: Anti-Spam ideas for usenet/list harvested email addresses"

    Relevant Pages

    • Re: Sudden increase in size of Debian?
      ... Do you already have a script to mirror only ... I think I would try only mirroring ... That attachment, debian.rsync, seems to just be anonftpsync ... specifically getting all the Packages files for the targets you want but ...
      (Debian-User)
    • Re: Sudden increase in size of Debian?
      ... > That attachment, debian.rsync, seems to just be anonftpsync (rsync ... > specifically getting all the Packages files for the targets you want ... another config file but for testing they are part of the script for now. ...
      (Debian-User)