RSS

Category Archives: backup

A Backup Script Mark 2

You may remember a while ago I posted about a backup script I wrote. I have put some work into it and improved it quite a bit. Here are some of the more important changes I made.

I realized that saying that it did incremental backups was not completely true. What it was doing was snapshots so I changes all of the terminology to snapshot instead of incremental.

Last time I mentioned that snapshots take a very long times. Well, I fixed that, kind of. Before the script was set to compress the tar files it generated which took hours if they were large. To illustrate this problem this problem I have some numbers. I ran a backup using the same exact 21.87GBs of data one compressed with bzip and one not. Using bzip the backup took 7h52m; without bzip the backup took 23m51s. That is 94.9% faster. All that time is for only a 1.26GBs (6.12% savings) of space saved. So I changed it so compression is not default anymore, though it is still an option. depending on the size and composition of your files to be backed up compression may still be worth using. Now I said I kind of fixed it because after I took those statistics I made the script hash the backup files so that when you can ensure that your backups are good. This takes some times, but no where near as much as compression. If you are a Mac OS X user you will have to install md5sum using macports or fink or whatever you want for this to work. Apple sucks and they can’t include this so you will have to install it yourself(if you already have macports you probably already have md5sum).

I added a option to restore from a snapshot (I believe the last version only supported restores from the normal backup).

You may have noticed a bug while using -v where if all you used was -v nothing would happen. Its fixed.

I added a paranoia mode that uses srm instead of rm. I hope that I can add more to this in the future. Encryption is still a problem.

I changed the configuration so that multiple users could be added. This part makes me with I hadn’t written this is bash. I hate the way it works and I can’t get anything better as I am pushing up against the barriers of the language (or my understanding of it?). check out the readme and /etc/backup/main.conf for more an this. Note that the install script will replace your current config file with a standard one(useless one) so make sure to save yours before you install.

If you want to look at all of the the changes take a look at the todo file.

Before I forget like last time it I need to link to this. So Here it is. I’m not sure how but this one is actually 2kb smaller than the last version(it is 30kb by the way).

I don’t know if I will continue to work on this or not. I am still using it even on leopard because I don’t know if I want to trust time machine. However there are some really great backup utilities out there so I may want to use one of those.

Just as last time if you have any suggestion, comments, or bug fixes you can leave them in the comments or email me.

Oh before I forget I’v been trying to think of a name for this (I’ve just been calling it backup so far). If you have any ideas I would love to hear them.

Advertisements
 
Leave a comment

Posted by on February 11, 2008 in backup, bash

 

Tags: , , ,

A backup script

Edit (nov. 17): I just realized I completely forgot to link to the file. Sorry. Here it is.


I got a external hard drive for Christmas last year to store some of the less important/larger things that I didn’t need or want on my internal 60GB hard drive. At some point I decided that it was impractical to continue backing up to DVDs and that I would have to use my external drive to to hold the backups unless I wanted to buy a tap drive (which I didn’t). So I manually copied my user file onto the drive every once in a while. I got really tired of how long that took so I started using rsync to manually backup the files. At the time I didn’t know anything about bash scripting but I though a shell script would be very useful so I didn’t have to type out the whole command (I’m lazy). I found the sites that I mentioned in the last site of the week and looked up what I needed to know to make a very simple shell script. I used that very simple script for several months until my brother started pushing me to add features and make something that other people might want to use. I’m glade he did this because as a result I learned a lot more about bash scripting and I have a much more functional and robust script. This script, after a few months of tweaking, is now at a point where I’m not embarrassed to let other people use it; It does everything that I want it to do (save 1 thing) and more importantly it is in complete working order.

So what does this script do?

  1. In place backups using rsync (as in it just keeps a folder up to date)
  2. Incremental backups
  3. backup of applications (I’m a mac user so I don’t think this would be a useful feature on Linux or BSD)
  4. restore

I bet your asking yourself “what is that feature that he wants but didn’t implement? and why didn’t he just go ahead and add it?” The feature is encryption (using gpg) and I didn’t add it because I couldn’t figure out how to get it to work in the way I want. For now if you want to encrypt it you will just have to do it by hand like a real man/woman.

If you have a large amount of data that needs to be back up it will take you a while so it is best to schedule the backup at night (if possible) or at any other time that the computer will be on but not in use. My user file is currently 22GB and I use an external USB Maxtor drive. With my setup it took me almost 9 hours for an incremental backup with a resulting file of 21.8GBs and 7 minuets for a normal+application backup. 21.8GBs is a a large file and only 1% smaller than the normal size of my user folder (assuming that it hasn’t changed in size significantly since I backed up last) so I may end up cutting out the compression in the future to save time. This talk of size brings me to another topic. My external drive is only 160GBs so I can’t fit very many backups on it which is the only reason there is an in-place backup in this script. In the future I may get a larger drive and remove the in-place backup.

I have only tested this on my computer (ibook G4) so I can’t guarantee that it will work on Linux. I am positive that it will work just fine on all Mac os X systems execpt that it uses bzip by default for incremental backups and mac os doesn’t come with bzip pre-installed, so I added an option (-g) to use gzip (which is installed by default.

There are a lot of different backup scripts and programs out there that may be far better than this one so feel free to stick with those if you want to. I don’t care how you backup your stuff as long as you do it. I’m sure you all know why it is important to back stuff up so I don’t think I need to get into that, however, I will say that just having a backup script/program is useless if you don’t use it. So Automate it!

Any bug reports/feature suggestions/patches can be left in the comments (in plain text please) or emailed to me (no I’m not going to give you my email address you have to find it yourself (I don’t need more spam.(by that I mean I don’t get spam and I don’t want to start(I’m not sure this is grammatically correct))))

 
Leave a comment

Posted by on November 13, 2007 in backup, bash

 

Tags: , , ,