The "verbose" Python

by Paul Agapow
in programming

tagged programming, unicode, text-handling, python

PYTHONPATH, PYTHONVERBOSE and other infuriations

The symptoms

“Suddenly” (as is traditional to commence these sort of stories), I started having problems with some of the Python installations on my local machine. As is the way of things, I had three distinct sources of Python:

  • The native / builtin Python that came with my Mac
  • The Python that was installed by brew to support other brew applications
  • The Pythons installed by anaconda

The symptoms were that some of these (notably anaconda) would complain about how they didn’t recognise PYTHONPATH or that they were seeing a package tree of a different version type (2.X) to their own version (3.x):

% python2.7
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D
% ~/anaconda/bin/python
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
Python 3.4.3 |Anaconda 2.1.0 (x86_64)| (default, Mar  6 2015, 12:07:41)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Similar symptoms would occur if you called various python-related utilities: pip, condo, etc.:

% conda list
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'

Setting the verbose flag on Python invocation showed that, indeed, the error was originating from within a Python 2.7 library that was being seen by a 3.x version Python:

...
# code object from /usr/local/lib/python2.7/site-packages/sitecustomize.py
# created '/usr/local/lib/python2.7/site-packages/__pycache__/sitecustomize.cpython-34.pyc'
# wrote '/usr/local/lib/python2.7/site-packages/__pycache__/sitecustomize.cpython-34.pyc'

Attempted solutions

  • PYTHONPATH was being set in my .bash_profile so I deleted it. Nope.
  • Maybe the compiled sitecustomize.py files had been corrupted? Delete them. Nope.
  • Uninstall the brew python? Nope.
  • Some of the error messages suggested unsetting PYTHONPATH. Nope.

Solution

Eventually this thread https://github.com/conda/conda/issues/448 gave me the clue. So I added this to my .bash_profile:

export PYTHONNOUSERSITE="$HOME/.local"

And then everything was fine.