# HOWTO use `pip` well

## tl;dr

*   Never use `sudo` with `pip`; really don't.
*   Always invoke as `python`_$M.$N_` -m pip ...` (or any other
    executable modules like Jupyter etc.).
*   Add _$HOME_`/.local/bin` at the start of the `PATH` variable
*   Move _$HOME_`/.local/{bin,lib}` to `/scratch/`_$USER_`/` and symlink.
*   Always specify `--user` with the `install` and `list` commands.
*   As a rule add `--upgrade` to the `install command`.
    in your `.profile` or equivalent.

Also read carefully the
[known PIP issues](https://gitlab.developers.cam.ac.uk/eng/divf/eng-divf-it-pages/blob/master/known/applications.md#python).

## Introduction

`pip` is a mechanism/"engine" and to operate it well it is a good idea
to have a mental model of how it works. As a very brief summary:

*   The `python3.9` interpreter binary contains a set of "paths" to its
    core libraries, and adds more paths for the site's library
    directories (under `/usr/local/`) and the user's library directories
    (under _$HOME_`/.local/`).
*   There must an exact match between the version of the Python
    interpreter and that of its libraries.
*   Moreover some modules contain C/C++/etc. code and their compiled
    versions must match exactly the Python interpreter version **and**
    the local CPU architecture.
*   The path of directories where here are executable commands that use
    Python must match that of the python modules.

## Description of `pip`

The `pip` module downloads various types of modules and writes them into
one of the library paths recognized by the Python interpreter binary
that runs it; if the module contains commands they are written into the
associated command path. Typical library paths and associated binary
paths:

*   Ubuntu 16.04:
    *   `/usr/lib/python3.5/dist-packages/`
    *   `/usr/bin/`
*   System-local:
    *   `/usr/local/lib/python`_M.N_`/dist-packages/`
    *   `/usr/local/bin/`_nameM.N_
*   User specific new style:
    *   _$HOME_`/.local/lib/python`_M.N_`/dist-packages/`
    *   _$HOME_`/.local/bin/`_nameM.N_
*   User specific old style:
    *   _$HOME_`/lib/python`_M.N_`/dist-packages/`
    *   _$HOME_`/bin/`_nameM.N_
*   DivF user specific and host specific:
    *   `/scratch/`_$USER_`/lib/python`_M.N_`/dist-packages/`
    *   `/scratch/`_$USER_`/bin/`_nameM.N_

That means that it is usually quite risky to invoke `pip` as "pip" or
"pip3" because it is not obvious which Python interpreter binary they
are associated with. It is best to make this explicit with using
something like:

    python3.9 -m pip

or sometimes if there are multiple versions of the Python 3.9
interpreter as:

    /usr/bin/python3.9 -m pip