Python requirements.txt: including other files, and installing specific git commits

Some notes on the lesser-known aspects of requirements.txt files.

Recursively including other requirements files

You can recursively include other requirements files like this:

$ cat requirements.txt 
-r client_requirements.txt
-r server_requirements.txt

$ cat client_requirements.txt 
mox==0.5.3
pexpect==3.3

$ cat server_requirements.txt 
pexpect==3.3

$ pip install -r requirements.txt

Unfortunately, this particular configuration won't actually work. You get:

Double requirement given: pexpect==3.3 (from -r server_requirements.txt (line 1)) (already in pexpect==3.3 (from -r client_requirements.txt (line 2)), name='pexpect')

pip can't actually handle complex dependency resolution, which means that if you have the same dependency in more than one file, even if the versions don't conflict, it will refuse to install anything.

Installing specific git commits

If you've ever had to work around a bug in a released version you might find yourself doing something like this:

git clone -b develop https://github.com/pyinstaller/pyinstaller.git
cd pyinstaller
git reset --hard edb5d438d8df5255a5c8f70f42f11f75aa4e08cf
python setup.py install

But pip can actually do all of that for you, this effectively does the same thing:

pip install git+https://github.com/pyinstaller/pyinstaller.git@edb5d438d8df5255a5c8f70f42f11f75aa4e08cf#egg=PyInstaller

and you can put that line into your requirements.txt:

$ cat requirements.txt
-r client_requirements.txt
-r server_requirements.txt
git+https://github.com/pyinstaller/pyinstaller.git@edb5d438d8df5255a5c8f70f42f11f75aa4e08cf#egg=PyInstaller

When this commit goes into a pip release, you'll also be able to specify install options inside requirements.txt.