Monday 13 January 2014

Paths, the route of nearly all evil

Generally speaking, if something isn't working on an OSX GIS it's a path issue. You can spend hours, days, months even, trying to get an installation to work only to discover that the path you thought was all set and correct is being overridden by another path setting.
Firstly, what is a path? A path is a list of addresses that the computer looks in when you tell it to run a programme or read a library. For instance, when you type "gdalinfo" into the terminal the computer looks at the first address in the list and looks for an executable file ("programme") called "gdalinfo". If it doesn't find it at the first address it moves on to the next address and so on.

To check which folders are currently "on your path" go to the terminal (you will find this under "Applications/Utilities" or use cmd-spacebar to bring up Spotlight and type "terminal" there), once in the terminal type "echo $PATH"

The system path


Which looks like this:

~ 16:53:37 $ echo $PATH
/Users/andrew/anaconda/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/MacGPG2/bin:/usr/texbin:/Library/Frameworks/GDAL.framework/Programs:/Library/Frameworks/PROJ.framework/Programs:/Library/Frameworks/PROJ.framework/Versions/4/PROJ:/Library/Frameworks/SQLite3.framework/Programs:/Users/andrew/Documents/BashScripts/gslib90/:/Applications/GRASS-6.4.app/Contents/MacOS/:/Users/andrew/Documents/BashScripts/
~ 16:53:45 $

In the path shown above each address is separated by a colon, thus "/Users/andrew/anaconda/bin" is the first address, "/usr/bin" is the second and so on.

From the image above you may be able to work out that "echo $PATH" means "print to the terminal the contents of the variable called PATH" in Bash. If you don't know it yet "Bash" is almost certainly the language you will use to communicate with the terminal in OSX (as well as Ubuntu and other Unix systems). There are other languages, all very similar, but Bash is the default in OSX.

There is another important path to be aware of: the python path. Python is a scripting language used by many programmes to add functions and aid interaction. It is also a very handy programming language in itself that you should consider learning. In the terminal type "python" to launch the Python interpreter.

Python and the Python path

As you can see, after starting Python I first imported a set of tools called "sys" using "import sys".
Then I asked sys to display the path that Python searches through. Like the system path shown by Bash, the path is one long line of text so instead I asked Python to loop through the path address by address so that the path becomes more readable. Type "quit()" to exit the Python interpreter.

Some of you may be wondering what the "anaconda" bit is all about. I can highly recommend Anaconda as a tidy, easy and easily controlled installation of Python and the many packages that you will need for scientific programming and GIS. Anaconda's website

So how do we set the path? Lets start with the system path, the Bash path.
There are three places where addresses are added to the Bash path permanently (i.e. not lost as soon as the terminal is exited). If you are comfortable using the terminal cd to "/private/etc"

Change directory to /private/etc/ and list the contents

Then type "ls" to list the contents of the folder (if you are wondering, I have set an alias in my .bash_profile so that ls runs with the -laF flags by default, e.g. alias ls='ls -laF'). Look through the list to find the two entries as below.

Path files in /private/etc/
If you would prefer to use Finder then go to the Finder menu bar and click on "Go" then enter the address "/private/etc".

Access a folder using Finder
Now scroll down the contents of the folder until you find "paths" and "paths.d".

Open the file called "paths" by right clicking on it in Finder and choosing "Open With TextEdit" or your text editor of choice. From the terminal you can open it using nano, vim or default editor, e.g. "nano paths" or "open paths".

The paths file in /private/etc/
TextWrangler (TextWrangler homepage) shows the file like this. Looking back at the path again:

etc 18:05:32 $ echo $PATH
/Users/andrew/anaconda/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/MacGPG2/bin:/usr/texbin:/Library/Frameworks/GDAL.framework/Programs:/Library/Frameworks/PROJ.framework/Programs:/Library/Frameworks/PROJ.framework/Versions/4/PROJ:/Library/Frameworks/SQLite3.framework/Programs:/Users/andrew/Documents/BashScripts/gslib90/:/Applications/GRASS-6.4.app/Contents/MacOS/:/Users/andrew/Documents/BashScripts/

we can recognise this list as the addresses listed after the Anaconda bin folder ("bin" stands for "binary" and implies compiled, executable programmes) and before the usr/X11/bin folder.
If you are still confused about the order be patient, all will become clear soon enough, just recognise that the order in which the addresses are listed here is the same as in the PATH variable.

The next thing to look at is the contents of the "paths.d" folder (directory).

The paths.d folder and its contents

So we can see from the image above that the paths.d folder contains 3 files, simple text files, each containing a single address and nothing more. Comparing their order in the folder with their order in the path we can see that the folder contents are read directly after the "paths" file and that the file name determines the order in the PATH.

So that's two places where the path is set. The third is in your .bash_profile. There is, of course, a fourth location, the system wide profile under "/etc/bashrc" but don't mess with this, it doesn't set any paths anyway, but it could in theory. You may even have a "launchd.conf" file under "/etc" in some older OSXs where the path may be meddled with. Way, way too much.

Lets get back to your .bash_profile, which is accessed most easily via the terminal or via TextWrangler and its ability to show and open hidden files. The name ".bash_profile" starts with a dot which means that the file is hidden from Finder and even from the terminal under normal circumstances.

In the terminal type "cd". This will move you to your home directory no matter where you currently are. Then type "ls". Unless you have already set an alias for ls to show hidden files your .bash_profile won't be listed. Enter "man ls" into the terminal. This opens the "man page" for the "ls" command. "man" is short for "manual" and is intended to be a usage guide for each bash command. Use the arrow keys to navigate down the man page. You should be able to find the following:

    -a      Include directory entries whose names begin with a dot (.).

Press "q" to exit the man page. Now enter "ls -a" into the terminal and it will list hidden files as well. If you prefer try "ls -al" to list all files on separate rows rather than one after the other.

If you do not have a .bash_profile you can easily create one using the terminal's own text editor. Enter "nano .bash_profile" into the terminal. If you have a .bash_profile already it will be opened if not, one will be created and opened for you. If the file is empty you may want to enter some text to explain what the file is, start by typing something along these lines:

The first lines of my .bash_profile




Always start a comment line (text that is to be read by humans and not the computer) with a hash, #
To save this, press ctrl-o to "write out" then ctrl-x to exit nano (note that nano uses ctrl not cmd).
You may continue to edit in nano if you wish or open the .bash_profile file with another editor ("open .bash_profile").

In the following example I set the path for the GDAL library as installed by the KyngChaos installer :

PATH=$PATH:/Library/Frameworks/GDAL.framework/Programs
export PATH

What these two lines say is "make the variable path the same as it is now plus the address /Library/Frameworks/GDAL.framework/Programs then export the PATH to the operating system". Approximately. If the first line had read:

PATH=/Library/Frameworks/GDAL.framework/Programs:$PATH

then the /Library/Frameworks/GDAL.framework/Programs address would be placed at the front of the list, i.e. it would be found first when reading the path.

We can also alter and export the path in a slightly more compact way.

export PATH="/Users/andrew/anaconda/bin:$PATH"

And that's it for the bash path. The Python path is a chapter in itself and I shall have to turn to that next time. Right now it's time for tea.

No comments:

Post a Comment