Re: Backing up suggestions?

On Sun, 20 Aug 2006, houghi <houghi@xxxxxxxxxxxxxxxxxx> wrote:-

David Bolt wrote:
There is quite a bit of self-discipline required for this back-up regime
because it means that as soon as I have copied the pictures from the
memory card, I have to ensure I have updated all my back-ups before I
can wipe the card and put it back in the camera.

This is where scripts get ito place very nicely.
Write a script called 'photobackup' that does all this for you. Using
zenity or kdialog, you can let the backup do its work and pop-up if you
need to interact, like connect the other PC, put the second or first dvd
in and so on.

I already have a few scripts that do a lot of the work for me. The part that imposes the self-discipline is the bit between copying the pictures from the memory card and making the backups. It's this point where I go through checking each individual picture and renaming them to something more recognisable than just the default img_1234.jpg[0].

After all the pictures have been renamed, I use another script that first checks the USB drive is present before starting the copying process[1]. It is only after this process is completed without errors does the script delete the images on the memory card.


Instead of kdialog, you can use zenity. Instead of cp, you can use rsync
and instead of taking my example, you can write something usefull. ;-)

For this job, I'd prefer not to use the GUI. Checking and sorting pictures does require use of a GUI but, for the actual renaming and archiving, I use a text console because it feels quicker.

Anyway, my present routine follows a nice sequence:

1, insert memory card into card reader;
2, plug in USB drive;
3, use script to copy files for temporary location ready for sorting;
4, identify pictures with the same subject, move them to another location and use my rename script to rename them;
5, once all the pictures have been renamed, move the pictures to their proper directories;
6, call script that builds par2 parity files (20% redundancy);
7, call script to perform archiving (see below);
8, once the archiving is completed, delete the images from the memory card;
9, unplug the USB drive and remove the memory card.

The archive script[4] performs four tasks:

1, checks if the USB drive is plugged in and aborts if not;
2, copies the images to the USB drive, using cp -ur[3];
3, uses rsync to copy the files to the other machines, using rsync -ac;
4, if all the archiving is completed correctly, delete the images on the memory card.

[0] I even have a script that takes a lot of the effort out of that. All I have to do is type " --name <whatever>" and it renames all the pictures in the current directory, using the name I specified. I even added a --dir option so it can rename them in other directories.

[1] There's a zero-byte file in the root directory of the drive that the archive script checks for. If it isn't found, the script assumes the drive isn't present.

[2] I've set up the rsync daemon on those machines so that I don't need to use a password to transfer the pictures. Instead all I need to use is $MACHINE_NAME::pictures/. Even though I don't expect any problems, I have ensured that access to that particular location using rsync is restricted to a single IP address.

[3] I use "ur", rather than "au" because the external drive is a FAT32 device which means that a couple of the options that 'a' enables aren't supported, namely preserving ownership and links.

[4] Here's my archive script. It is very specific to my system and network, but can quite easily be adapted to take different options on the command line. I guess that someone could even add eye candy by using the GUI:


# check if USB drive is plugged in
if [ ! -e "/media/WD_USB_2/present" ]
echo "Plug the USB drive in and start again"
exit 1

# USB drive is present, so perform the back-up
cp -ur "/local/pictures" "/media/WD_USB_2/" || exit 1

# Backed up to USB drive. Now to machine 1
rsync -ac "/local/pictures/"* || exit 1

# Backed up to adder. Now to machine 2
rsync -ac "/local/pictures/"* || exit 1

# Done the back-ups, now clear the memory card
find "/media/EOS_DIGITAL" -type f -name "*.jpg" -print0 | xargs -0 rm
find "/media/EOS_DIGITAL" -type f -name "*.JPG" -print0 | xargs -0 rm
find "/media/EOS_DIGITAL" -type f -name "*.crw" -print0 | xargs -0 rm
find "/media/EOS_DIGITAL" -type f -name "*.CRW" -print0 | xargs -0 rm

# we're done
echo "Back-up complete. Remove the USB drive and memory card now."

exit 0

David Bolt

Member of Team Acorn checking nodes at 50 Mnodes/s:
AMD1800 1Gb WinXP/SUSE 9.3 | AMD2400 256Mb SuSE 9.0 | A3010 4Mb RISCOS 3.11
AMD2400(32) 768Mb SUSE 10.0 | Falcon 14Mb TOS 4.02 | A4000 4Mb RISCOS 3.11
AMD2600(64) 512Mb SUSE 10.0 | | RPC600 129Mb RISCOS 3.6