Installing Python packages in $HOME

On Wed 08 May 2013 | By Raphaël Barrois | Category : Python | Tags : Python

So you've become quite familiar with some Python-developed tools, and have installed them on all your computers.

For a new project, you get an account on a new server; alas, your favorite tools isn't installed there. And you don't have root access. How can you remedy to this issue?

Note

This recipe also applies to installing packages on your personal laptop, and should be preferred over sudo pip install foo.

First, check if the sysadmin can install the distribution-provided package for your program. This is the sanest way to ensure updates to that tool are properly installed, and increases the chances to get your tool installed on other hosts.

If that's not possible, you'll have to install this by hand.

Please, avoid virtualenv here

If you're a Python developer, chances are you're already utilizing virtualenv a lot:

  • It allows to split development environments, making dependency tracking easier
  • It means that broken dependencies can be installed without polluting the system
  • It doesn't require root access to install new packages

Nevertheless, it's not exactly the right tool for this issue:

  • Virtualenvs cannot be stacked, which means that you would have to install your tool in each and every virtualenv you create. virtualenvwrapper might help through its hook system, but we'd be getting into dark magic for a simple use case
  • You may wish to use the tool without having to call . ~/.venvs/foo/bin/activate everytime

Enter install --user

The key solution to all this is the --user option to pip and easy_install install command:

$ pip install --user uconf
$ ls ~/.local/bin
uconf*

Once your package has been installed to ~/.local, you can make it available by inserting the following lines to your .zshrc / .bashrc:

export PYTHONPATH="${PYTHONPATH}:${HOME}/.local/lib/python2.7/site-packages"
export PATH="${PATH}:${HOME}/.local/bin"

Rationale

A common question regarding this scheme is: "Why should I install packages into $HOME instead of sudo + system-wide"?

A basic answer is: "You won't always have root" (and using exploits is not an option here).

A more realistic reason is that the Use standard ways of installing tools into custom locations process can be ported to other languages / frameworks easily, thus meaning that you only have one place for local tools: .local.

Actually, this has always been the way to experiment with compiled programs; why should compiled programs be any different?

Other articles

Patching datetime.date.today() across Python implementations

On Sun 21 April 2013 | By Raphaël Barrois | Category : Python | Tags : Python

Altering builtin functions and classes of Python during tests if often frowned upon, but can be actually required while writing unit tests.

The typical builtin requiring such alterations are the datetime.date.today and datetime.datetime.now() functions, which return (obviously) time-dependent values.

We'll take a look at this ...

More…

Setting up a private, team-wide PyPI repository

On Wed 29 August 2012 | By Raphaël Barrois | Category : Python | Tags : System ~ Python

When developing Python applications, it may be useful to store some applications in a private repository.

This provides several benefits:

  • Dependencies available even if PyPI and its mirrors are down
  • Storing custom forks of upstream packages
  • Providing private packages in a standardized manner.

For this purpose, three components are required ...

More…

Getting Firefox/Chrome to use system-wide certificates

On Mon 13 August 2012 | By Raphaël Barrois | Category : Linux | Tags : System ~ Web ~ Linux ~ Gentoo

While browsing the web, I was getting quite annoyed that firefox, chrome and console tools (wget, ...) each maintained a separate list of trusted certificates.

Which led me to dive into the arcane world of NSS, the "Network Security Services".

Note

This is a rather long post, which will walk you ...

More…

Xel.Mind — Random thoughts

On Fri 10 August 2012 | By Raphaël Barrois | Category : Misc | Tags : Me

After a couple of years spent thinking about "Well, I should write out stuff somewhere, for my future self", I've finally crossed the line.

What can you expect on this "Weblog"?

  • Random thoughts on technology
  • Articles around Python, Gentoo, Django, SysAdmin stuff, …
  • Every once in a while, considerations on ...

More…