Today a friend and I were working on installing a Python
application on a server that sat behind an HTTP proxy.
The first step was figuring out how to tell
to use an http proxy. This step was simple, we
just had to add the proxy parameter on the commandline, like so:
$ pip install -r requirements.txt --proxy=<proxy address>
The installation seemed to be proceeding correctly, with
telling us files were getting downloaded succesfully from pypi.python.org.
Unfortunately about halfway through the installation
pip aborted with the following message:
File "/usr/local/lib/python2.7/site-packages/`pip`-1.5.5-py2.7.egg/pip/_vendor/requests/adapters.py", line 387, in send raise Timeout(e) Timeout: (<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x1f70650>, 'Connection to pypi.python.org timed out. (connect timeout=15)')
We reasoned there had to be something wrong with either our environment or with the proxy and thus we proceeded to waste the following hour debugging and checking different parts of the system for problems.
Imagine our surprise when we decided to search online and
ran into this bug: https://github.com/pypa/pip/issues/1805.
Apparently this is a know issue in
requests seems to use
a pool of HTTP connections to download the different
packages from the internet. When there is a large number
of packages, new connections in the pool are created, however
they do not seem to inherit the proxy settings. The result
is that the connection times out. This was discovered
by github user alexandrem.
In his comment inside the bug he proposes a small fix:
Fix for timeout problem
Open file: ../lib/python2.6/site-packages/pip/_vendor/requests/adapters.py Change line 209 to the following: if True or not proxy in self.proxy_manager:
This effectively forces
pip to instantiate a new
every time, but it allows the installation to complete. I thought it
was a fair tradeoff, with no obvious side-effects.
There is a fix already in the requests package, but it doesn't seem
to have reached the latest
pip. Until it does, you can use the previous
fix, to allow your installations to complete succesfully.