environment variables - How to add a directory to the PATH?

How do I add a directory to the $PATH in Ubuntu and make the changes permanent?

A path set in .bash_profile will only be set in a bash login shell (bash -l).
If you put your path in .profile it will be available to your complete desktop session. That means even metacity will use it.

For example ~/.profile:

if [ -d "$HOME/bin" ] ; then
  PATH="$PATH:$HOME/bin"
fi

Btw, you can check the PATH variable of a process by looking at its environment in /proc/[pid]/environ (replace [pid] with the number from ps axf). E.g. use grep -z "^PATH" /proc/[pid]/environ

Note:

bash as a login shell doesn't parse .profile if either .bash_profile or .bash_login exists. From man bash :

it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that
order, and reads and executes commands from the first one that exists
and is readable.

See the answers below for information about .pam_environment, or .bashrc for interactive non-login shells, or set the value globally for all users by putting a script into /etc/profile.d/ or use /etc/X11/Xsession.d/ to affect the display managers session.