RSS

Category Archives: bash

Adding some color to ls in Mac OS

If you have ever used ls in the Mac OS terminal you have probably noticed that you get a boring monochrome list. If you have never used Linux before you might not find this a problem at all. Why would you anyone expect or need it to have color? Well, sometimes color is very important, for instance if you are in an unfamiliar directory and you don’t know what is a file and what is a sub-directory.If you are completely new to the Terminal/Unix-like operating systems a short explanation of what ls is is in order.ls is a program which lists all of the files and directories in a directory. Try opening the terminal and typing ls to see what I mean (type man ls for more information on how to use ls). On a mac, and all other BSDs to my knowledge, the output text will all be in one color (probably black in your case),but on Linux the output will be color coded (files and directories have different color text)So you want Mac OS to have colored output just like Linux? well, that is easy. ls has the argument -G which will output colored text (on Mac OS. Linux uses -C (–color) but you shouldn’t need that, and I don’t know what other BSDs do.) So, you could type ls -G every time you want to list the contents of a directory, but that would get every annoying after a while. There is an easy way you can make it output colors every time you run ls so you won’t have to add the -G every time! Go the command line and type:

cd ~/ (if you are not already there)

echo alias ls=’/bin/ls -G’ >> .bash_profile

Now every time you type ls you will see colored output (if you ever want to remove this just edit ~/.bash_profile with your favorite text editor and remove the line we just added) Enjoy!

 
Leave a comment

Posted by on December 21, 2008 in apple, bash, concatenation

 

Tags: , , , , , , , ,

Useful Commandline Tools

These are some of the most useful, commonly used, or coolest command line tools that I know of. That doesn’t mean that you will find them useful or use them much; I can only go off of what I know and that is what I do myself :).

You should be able to find most of these pre-installed on any *nix (including mac os).

1. wget
wget is a download manager and is by far my favorite download manager. Some people prefer curl, but I don’t. The most basic usage of wget is just wget (url).

2. rsync
rsync is a very simple but powerful tool. If you aren’t using it you probably should be. It simply “syncs” two folders, meaning that all files from one folder will be copied to another folder (this is a simplistic explanation). The most basic usage you will probably need is rsync -av source/ destination/. This will move any files from source/ that are not already in destination/ to destination/ and replace any files in destination/ that are in both but differ(say destination/ has an old version it will be replaced by the one from source/), note that the v flag is for verbose and may be left off. rsync -av –delete source/ destination/ may be used if you want the contents of destination/ to be exactly the sam as source/.

3. lsof
lsof lists all open files. So want to see what your users are doing? No problem just type lsof as root.

4. history
Ok, you really should know this one. It list terminal history. So you forgot that command you used yesterday to do something, just type history into bash and see every command you have an in the last however long your history is set to store.

5. link/ln
linux links two files together. The syntax is linux source Target. I don’t want to get into the whole hard v symbolic links right now so I will just quote the man page ” A hard link to a file is indistinguishable from the original directory entry; any changes to a file are effectively independent of the name used to reference the file.” ” A symbolic link contains the name of the file to which it is linked. The referenced file is used when an open(2) operation is performed on the link. A stat(2) on a symbolic link will return the linked-to file; an lstat(2) must be done to obtain information about the link. The readlink(2) call may be used to read the contents of a symbolic link.”
note that link and ln are the same thing.

6. periodic
nixen have scripts that they need to run on a periodic bases to keep your system running well. If you don’t leave your computer on 24 7 some of them won’t be ran. If you don’t leave your computer on all the time to keep your system in good heath it is a good idea to run sudo periodic -v daily weekly monthly

7. scp
scp is cp over ssh. It is really nice, trust me. To use it type scp source destination in the terminal. scp is meant to transfer files from one computer to another so either the source or destination will need to look something like username@ipaddress:path. You will also need the user password to transfer files, of course.

8. at
at allows you to run a command in the future. I don’t really use it much and I’m not 100% sure how it all works and its late so I’m going to have to just refer you to the man page. (this is one of the ones that is just cool)

9. clear
clear clears the terminal of all your old garbage. So if you have been working for a while and you want a clean working area just type clear.

For more on any of these refer to your friendly system man (as in man (name of program)…).

 
1 Comment

Posted by on May 31, 2008 in bash, Commandline, linux

 

Tags: , , , , , , , , , , ,

A Backup Script Mark 2

You may remember a while ago I posted about a backup script I wrote. I have put some work into it and improved it quite a bit. Here are some of the more important changes I made.

I realized that saying that it did incremental backups was not completely true. What it was doing was snapshots so I changes all of the terminology to snapshot instead of incremental.

Last time I mentioned that snapshots take a very long times. Well, I fixed that, kind of. Before the script was set to compress the tar files it generated which took hours if they were large. To illustrate this problem this problem I have some numbers. I ran a backup using the same exact 21.87GBs of data one compressed with bzip and one not. Using bzip the backup took 7h52m; without bzip the backup took 23m51s. That is 94.9% faster. All that time is for only a 1.26GBs (6.12% savings) of space saved. So I changed it so compression is not default anymore, though it is still an option. depending on the size and composition of your files to be backed up compression may still be worth using. Now I said I kind of fixed it because after I took those statistics I made the script hash the backup files so that when you can ensure that your backups are good. This takes some times, but no where near as much as compression. If you are a Mac OS X user you will have to install md5sum using macports or fink or whatever you want for this to work. Apple sucks and they can’t include this so you will have to install it yourself(if you already have macports you probably already have md5sum).

I added a option to restore from a snapshot (I believe the last version only supported restores from the normal backup).

You may have noticed a bug while using -v where if all you used was -v nothing would happen. Its fixed.

I added a paranoia mode that uses srm instead of rm. I hope that I can add more to this in the future. Encryption is still a problem.

I changed the configuration so that multiple users could be added. This part makes me with I hadn’t written this is bash. I hate the way it works and I can’t get anything better as I am pushing up against the barriers of the language (or my understanding of it?). check out the readme and /etc/backup/main.conf for more an this. Note that the install script will replace your current config file with a standard one(useless one) so make sure to save yours before you install.

If you want to look at all of the the changes take a look at the todo file.

Before I forget like last time it I need to link to this. So Here it is. I’m not sure how but this one is actually 2kb smaller than the last version(it is 30kb by the way).

I don’t know if I will continue to work on this or not. I am still using it even on leopard because I don’t know if I want to trust time machine. However there are some really great backup utilities out there so I may want to use one of those.

Just as last time if you have any suggestion, comments, or bug fixes you can leave them in the comments or email me.

Oh before I forget I’v been trying to think of a name for this (I’ve just been calling it backup so far). If you have any ideas I would love to hear them.

 
Leave a comment

Posted by on February 11, 2008 in backup, bash

 

Tags: , , ,

A backup script

Edit (nov. 17): I just realized I completely forgot to link to the file. Sorry. Here it is.


I got a external hard drive for Christmas last year to store some of the less important/larger things that I didn’t need or want on my internal 60GB hard drive. At some point I decided that it was impractical to continue backing up to DVDs and that I would have to use my external drive to to hold the backups unless I wanted to buy a tap drive (which I didn’t). So I manually copied my user file onto the drive every once in a while. I got really tired of how long that took so I started using rsync to manually backup the files. At the time I didn’t know anything about bash scripting but I though a shell script would be very useful so I didn’t have to type out the whole command (I’m lazy). I found the sites that I mentioned in the last site of the week and looked up what I needed to know to make a very simple shell script. I used that very simple script for several months until my brother started pushing me to add features and make something that other people might want to use. I’m glade he did this because as a result I learned a lot more about bash scripting and I have a much more functional and robust script. This script, after a few months of tweaking, is now at a point where I’m not embarrassed to let other people use it; It does everything that I want it to do (save 1 thing) and more importantly it is in complete working order.

So what does this script do?

  1. In place backups using rsync (as in it just keeps a folder up to date)
  2. Incremental backups
  3. backup of applications (I’m a mac user so I don’t think this would be a useful feature on Linux or BSD)
  4. restore

I bet your asking yourself “what is that feature that he wants but didn’t implement? and why didn’t he just go ahead and add it?” The feature is encryption (using gpg) and I didn’t add it because I couldn’t figure out how to get it to work in the way I want. For now if you want to encrypt it you will just have to do it by hand like a real man/woman.

If you have a large amount of data that needs to be back up it will take you a while so it is best to schedule the backup at night (if possible) or at any other time that the computer will be on but not in use. My user file is currently 22GB and I use an external USB Maxtor drive. With my setup it took me almost 9 hours for an incremental backup with a resulting file of 21.8GBs and 7 minuets for a normal+application backup. 21.8GBs is a a large file and only 1% smaller than the normal size of my user folder (assuming that it hasn’t changed in size significantly since I backed up last) so I may end up cutting out the compression in the future to save time. This talk of size brings me to another topic. My external drive is only 160GBs so I can’t fit very many backups on it which is the only reason there is an in-place backup in this script. In the future I may get a larger drive and remove the in-place backup.

I have only tested this on my computer (ibook G4) so I can’t guarantee that it will work on Linux. I am positive that it will work just fine on all Mac os X systems execpt that it uses bzip by default for incremental backups and mac os doesn’t come with bzip pre-installed, so I added an option (-g) to use gzip (which is installed by default.

There are a lot of different backup scripts and programs out there that may be far better than this one so feel free to stick with those if you want to. I don’t care how you backup your stuff as long as you do it. I’m sure you all know why it is important to back stuff up so I don’t think I need to get into that, however, I will say that just having a backup script/program is useless if you don’t use it. So Automate it!

Any bug reports/feature suggestions/patches can be left in the comments (in plain text please) or emailed to me (no I’m not going to give you my email address you have to find it yourself (I don’t need more spam.(by that I mean I don’t get spam and I don’t want to start(I’m not sure this is grammatically correct))))

 
Leave a comment

Posted by on November 13, 2007 in backup, bash

 

Tags: , , ,

Site of the Week: Bash Scripting

Well, its Friday and that means it is time for another site of the day. This week I actually have two sites for you because I felt like it. So as you have probably already figured of from the title today I’m going to be giving you sites about bash scripting. The first is the site that I used to learn how to script the second is the site that I use for reference; I haven’t really read through it but I would like too sometime.

Linuxcommand.org is actually just about bash in general but it has a pretty good introduction to bash shell scripting. I used this site to learn because it does a very good job of presenting its information in a well organized, good looking, easy to understand way. If all you want is a introduction to shell scripting this site is great.

Linux Shell Scripting Tutorial: A Beginner’s handbook offers a much more in depth introduction to shell scripting but it isn’t as pretty and may be a little more complex than a noob would want. I use this book mostly for reference because it goes into more detail and a wider range of topics than Linuxcommands does. I probably should read this so I can learn about all the cool stuff that I that I didn’t learn with Linuxcammands.

Bash shell scripting is a great way to introduce someone to the basic concept of programming. If you want to learn to program but don’t think you can handle a real programing language or if you have kids that want to learn how to program, Bash scripting is probably the best way to go.

I think now I should point out the differences between scripting and programming. They are actually pretty simple. Scripting is writing a list of commands for programs to carry out while programming is writing code that will be compiled/interpreted and is itself ran, not just telling other things what to do but doing them itself. I don’t know if that is very clear or at all a good way of explaining the difference. If you can do a better job the comments awaits you.

 
1 Comment

Posted by on October 26, 2007 in bash, site of the week

 

Tags: , ,

Concatenation: Command Line Tools

I’ve been working on this for a while now and I think I should finally put it out but it isn’t quite finished. I just wrote a 500 paper for school (today was the first day of school mind you) so I don’t feel like puting in the time it would take for me to finish this and I’m even havening trouble typing the letters of these words in order so I’m just going to but it out the way it is and maybe fix it later. not like it matters because no one will ever read it.

This is the first and possibly the only post in a concatenation series. I have greped a few (ok two) other blogs and found some useful list of the best command line tools and I’m going to put them here mostly so they will be in one place so I can get easy access to them but if they help you and you want to believe I have done this out of the goodness in my heart for your sole benefit then that is ok too. Make sure to check out the blogs I’ve pulled this stuff from for some a more detailed description on these commands.
The first up comes from my favorite blog
Ten Cool Coreutils commands
1. tac
tac works just like cat but reverse the text in the output. I’ve personally never used it but it might come in handy some time.
2. tee
tee prints is used to print output of another command both to the standard output (screen) and a file.
3. pr
formats text for printing
4. stat
this is used to get information on a file
5. yes
yes infinitely prints y or any arg you give it. It Also has the added benefit for using 100% of your processor.
6. expand
7. split
8. uiq
9. wc
10. shred

The other comes from a blog that I have never actually read but I found it on Digg . I may start reading it.
Ten OS X Command Line Utilities you might not know about
1. ssh
2. top
3. lsbom
4. say
5. softwareupdate
6. ifconfig
7. lipo
8. screencapture
9. fink
10. darwinports

If you know of any other blogs that have good lists let my know and I will post them here.

 
Leave a comment

Posted by on September 18, 2007 in bash, concatenation