github soundcloud rss
python package management
Sep 27, 2017
2 minutes read

I follow some simple rules for Python packages.

  • Never touch the OS site-packages with pip
  • Never run pip with escalated privileges (sudo)
  • Always use a local ($HOME) pip
  • Use the OS package manager when installing to site-packages to avoid competing package managers (pip and yum, for example)

Where’s site-packages located?

python3 -m site
python3 -m user-site

Read up on the Python docs and the pip docs to get a better understanding.

how to win

Assuming you have no traces of pip left.

Grab get-pip.py from the official pip website and install it under $HOME:

python3 get-pip.py --user

After this command succeeds, a pip binary will be placed in ~/.local/bin. Make sure you have this folder in your $PATH:

export PATH=$HOME/.local/bin:$PATH

Once pip is installed and set up we will install only 2 packages in the user-environment: virtualenv and pipsi.

pip install virtualenv --user

Grab pipsi from GitHub and install it:

python3 get-pipsi.py

pipsi installs each package into ~/.local/venvs/PKGNAME and then symlinks the scripts (CLI entry points) into ~/.local/bin. This makes it so you don’t have to activate the virtual environments manually to use the programs.

We can use pipsi to install pew (a virtualenv manager):

pipsi install pew

overview

pipsi --version
pip -V

jupyter notebooks

I like to follow the same principles for Jupyter notebooks.

pipsi install notebook

Create a virtualenv that can be used as a notebook kernel:

pew new pandas3 --python=python3
pip install ipykernel
pip install pandas

Create the kernel spec:

ipykernel install --user --name 'python3-pandas' --display-name 'Python 3 (venv pandas3)'

It will be placed in ~/.local/share/jupyter/kernels/python3-pandas/kernel.json

Whenever you want to use pandas in Jupyter you create a new notebook with this kernel.

other

I use the OS package manager to install some packages, for example python3-neovim and python2-neovim in the Debian repos. Hopefully they are optimized to work with the neovim package.

inspiration

Hitchhiker’s guide to Python

Now, tell me how to do it even better.


Back to posts


comments powered by Disqus