Re: Backing up suggestions?



On Sun, 20 Aug 2006, houghi <houghi@xxxxxxxxxxxxxxxxxx> wrote:-

David Bolt wrote:
There is quite a bit of self-discipline required for this back-up regime
because it means that as soon as I have copied the pictures from the
memory card, I have to ensure I have updated all my back-ups before I
can wipe the card and put it back in the camera.

This is where scripts get ito place very nicely.
Write a script called 'photobackup' that does all this for you. Using
zenity or kdialog, you can let the backup do its work and pop-up if you
need to interact, like connect the other PC, put the second or first dvd
in and so on.

I already have a few scripts that do a lot of the work for me. The part that imposes the self-discipline is the bit between copying the pictures from the memory card and making the backups. It's this point where I go through checking each individual picture and renaming them to something more recognisable than just the default img_1234.jpg[0].

After all the pictures have been renamed, I use another script that first checks the USB drive is present before starting the copying process[1]. It is only after this process is completed without errors does the script delete the images on the memory card.

<Snippety>

Instead of kdialog, you can use zenity. Instead of cp, you can use rsync
and instead of taking my example, you can write something usefull. ;-)

For this job, I'd prefer not to use the GUI. Checking and sorting pictures does require use of a GUI but, for the actual renaming and archiving, I use a text console because it feels quicker.

Anyway, my present routine follows a nice sequence:

1, insert memory card into card reader;
2, plug in USB drive;
3, use script to copy files for temporary location ready for sorting;
4, identify pictures with the same subject, move them to another location and use my rename script to rename them;
5, once all the pictures have been renamed, move the pictures to their proper directories;
6, call script that builds par2 parity files (20% redundancy);
7, call script to perform archiving (see below);
8, once the archiving is completed, delete the images from the memory card;
9, unplug the USB drive and remove the memory card.

The archive script[4] performs four tasks:

1, checks if the USB drive is plugged in and aborts if not;
2, copies the images to the USB drive, using cp -ur[3];
3, uses rsync to copy the files to the other machines, using rsync -ac;
4, if all the archiving is completed correctly, delete the images on the memory card.


[0] I even have a script that takes a lot of the effort out of that. All I have to do is type "namepics.sh --name <whatever>" and it renames all the pictures in the current directory, using the name I specified. I even added a --dir option so it can rename them in other directories.

[1] There's a zero-byte file in the root directory of the drive that the archive script checks for. If it isn't found, the script assumes the drive isn't present.

[2] I've set up the rsync daemon on those machines so that I don't need to use a password to transfer the pictures. Instead all I need to use is $MACHINE_NAME::pictures/. Even though I don't expect any problems, I have ensured that access to that particular location using rsync is restricted to a single IP address.

[3] I use "ur", rather than "au" because the external drive is a FAT32 device which means that a couple of the options that 'a' enables aren't supported, namely preserving ownership and links.

[4] Here's my archive script. It is very specific to my system and network, but can quite easily be adapted to take different options on the command line. I guess that someone could even add eye candy by using the GUI:

#!/bin/bash

# check if USB drive is plugged in
#
if [ ! -e "/media/WD_USB_2/present" ]
then
beep
echo "Plug the USB drive in and start again"
exit 1
fi

# USB drive is present, so perform the back-up
#
cp -ur "/local/pictures" "/media/WD_USB_2/" || exit 1

# Backed up to USB drive. Now to machine 1
#
rsync -ac "/local/pictures/"* adder.davjam.org::pictures/ || exit 1

# Backed up to adder. Now to machine 2
#
rsync -ac "/local/pictures/"* cobra-mk3.davjam.org::pictures/ || exit 1

# Done the back-ups, now clear the memory card
#
find "/media/EOS_DIGITAL" -type f -name "*.jpg" -print0 | xargs -0 rm
find "/media/EOS_DIGITAL" -type f -name "*.JPG" -print0 | xargs -0 rm
find "/media/EOS_DIGITAL" -type f -name "*.crw" -print0 | xargs -0 rm
find "/media/EOS_DIGITAL" -type f -name "*.CRW" -print0 | xargs -0 rm

# we're done
#
beep
echo "Back-up complete. Remove the USB drive and memory card now."

exit 0


Regards,
David Bolt

--
Member of Team Acorn checking nodes at 50 Mnodes/s: http://www.distributed.net/
AMD1800 1Gb WinXP/SUSE 9.3 | AMD2400 256Mb SuSE 9.0 | A3010 4Mb RISCOS 3.11
AMD2400(32) 768Mb SUSE 10.0 | Falcon 14Mb TOS 4.02 | A4000 4Mb RISCOS 3.11
AMD2600(64) 512Mb SUSE 10.0 | | RPC600 129Mb RISCOS 3.6
.