SUMMARY: rsync, fat32, firewire

From: Andy Rowan (bogey2521_at_hotmail.com)
Date: 03/07/05

  • Next message: Bob Alexander: "Can I go back to a previous package level ??" 
    Date: Mon, 07 Mar 2005 12:21:09 -0500
To: debian-user@lists.debian.org

    Got my mixed-platform backup system running, and everything seems to be
    peachy. I learned a few lessons, so I thought I'd compile them here for
    posterity. None of these are anything new to an expert, but they may be to
    others.

    What I'm doing is using rsync over the LAN to do backups of some windows
    2000 machines and some unix/linux machines, onto a dedicated backup server
    that runs Debian Sarge. It has a couple of large IDE drives (250GB each),
    and a couple of external (firewire/USB) hard drives that I take offsite on
    a rotating basis. I'm using rsync running locally to bring the external
    drives up to date with the latest generations of the backups that are
    stored on the internal drives. Also on the internal drives are old
    generations of the rsync backups, which are created by use of "cp -al" so
    that files that don't change from one backup to the next are just done as
    multiple hard links to the same file. Result is that each backup job after
    the first one is just an incremental backup in terms of what gets
    transferred and how much additional disk space is consumed on the backup
    server. But any one of the backup generations appears as if it were a full
    backup because of the hard links. Nightly backup runs on some of the
    clients are measured in seconds, and I'll never have to run a full backup
    again unless something croaks.

    Things I learned along the way.

    FAT32 works better with linux than NTFS does. I wanted to be able to take
    the external hard drives and connect them to a windows machine for
    restores, so I went with FAT32, based on advice here and elsewhere that
    writing to an NTFS partition in linux is risky.

    When you use rsync to sync an ext3 partition onto a FAT32 partition, a
    couple of things happen. One is that the FAT32 partition isn't going to
    like certain filename characters that are ok in the ext3 partition ... like
    filenames with ":" in them. The result is that files coming off of the
    linux/unix servers can't be stored correctly on the FAT32. My solution to
    this was to split the external drives into two partitions, one FAT32 for
    the windows backups and one ext3 for the linux/unix ones. Another option
    would have been to go ahead and use ext[23] and rely on a software solution
    that would allow a windows machine to access them.

    In order to create the FAT32 partitions, I used mkfs on the linux
    machine. Windows 2000 is perfectly happy USING a large fat32 partition,
    but if you try to create one, it runs through what appears to be an entire
    format process, and then fails at the end with a message that the
    filesystem is too large. Even the windows help says that this is by
    design, and that it's got nothing to do with inherent limits to fat32, it's
    just to encourage (also known as force) you to use NTFS instead. Why they
    go ahead and let the program run through 99% of a time-consuming process
    before failing rather than just having it tell you right away is left as an
    exercise to the reader.

    Another lesson is that the FAT32 partition should be mounted with the
    "shortname=mixed" option. If not, the rsync gets messed up. The default
    option is "lower," which means that files and directories with short names
    will have their names forced to lower case. But then rsync will think
    they're different from the ones it sees in the source that are upper case,
    and it will send the upper case ones again, and then delete the lower case
    ones. But since it's really the same files, the end result is it transfers
    a bunch of data and then deletes it all! Not good.

    USB 2 works better than firewire. The external drives have both
    interfaces. When I used firewire, I was getting errors in the data
    transfer, and losing a lot of data. Using USB, this didn't happen.

    Easiest way to set up the windows backup clients seemed to me to be to
    install cygwin, rsync, and ssh. A shell script takes care of the client
    side of the rsync transaction, and the backup server runs the rsync daemon
    via ssh. Then on the windows client, I wrote a batch file to launch the
    shell script, and installed that as a scheduled task. No messing with
    services on the windows end that way.

    Now in hindsight, the backuppc package would probably have saved me some of
    this trouble, but I like the way this turned out in the end.

    -Andy
    (This hotmail address may go away, once the spammers catch up to it. A
    more permanent address is rowan at crssa rutgers edu, with dots and a real
    at sign in the right places.) 

    -- 
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
  • Next message: Bob Alexander: "Can I go back to a previous package level ??"

    Relevant Pages

    • Re: Confused over CIFS
      ... machine I call "Ubuntu" is used to administrate and I use it to make ... nervous about rsync because the files and numberous and small -- even ... which is sharing its Windows ... All of your Windows XP boxes backup to that machine. ...
      (Ubuntu)
    • Re: Backup Image for my needs
      ... I will store on windows box connected ... My backup consists of just copying the 'files' to a backup medium ... Here are two links on how to create backups with rsync. ... type 'man command' on the command line. ...
      (Debian-User)
    • Re: [SLE] Windows backups to a Linux server
      ... files he wants to backup should be put somewhere else. ... rsync is still your answer. ... From the linux box write a 1 line script or just ... attributes from windows to unix. ...
      (SuSE)
    • RE: Backup server with Windows Clients
      ... I use Rsync to do this very thing. ... Backup server with Windows Clients ... To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org with a subject of "unsubscribe". ...
      (Debian-User)
    • Re: Hard drive format compatibility - what am I missing?
      ... Sorry for the omission - that's Windows XP HE. ... >I'm trying to help out someone with setting up a backup procedure. ... >I created a primary partition on the external drive using the laptop, ... >for the FAT32 partition? ...
      (microsoft.public.windowsxp.hardware)