Pipenv: A Better Way

Pipenv: A Better Way?

Many people prefer a tool called pipenv instead of using pip and virtualenv seperately. I include pipenv here and leave it up to you to decide which you like better. Perhaps you work in a team where pip + virtualenv is the norm, that's OK. Maybe you want to convince them to use pipenv instead after reading this!

Why pipenv?

Using pipenv has a number of advantages compared to using pip and virtualenv seperately. These are the main ones:

  • You no longer need to use pip and virtualenv separately. Instead, you have one tool that does it all --- and more!
  • pipenv separates your top-level dependencies from the last tested combination (e.g. the output of pip freeze). This makes dependency management more user-friendly for you as a developer.
  • pipenv encourages the use of the latest versions of dependencies to minimize security risks. It can even scan your dependencies for known vulnerabilities.
  • pipenv gives insight into your dependency graph with pipenv graph.
  • pipenv hashes all dependencies. It will detect packages that have been tampered with after you initially included it as a dependency.
  • pipenv can work with requirements.txt files too. If there is one, it will automatically detect it and convert it into a Pipfile.

Perhaps some of these points confuse you. Don't worry, they will be explained!

Installing pipenv

If you are on MacOS or Linux, I strongly recommend Homebrew. If you are not using Homebrew already, please see the bonus chapter [Bonus: Using Homebrew (MacOS and Linux)] and install it. After that, it’s as simple as:

$ brew install pipenv

Alternatively, chances are you have pip installed as a system-wide package already. I do this on my systems. Sure, I use virtual environments for all my projects. But I also installed a couple of tools like pip as system-wide packages. If you have pip installed, simply use it to install pipenv:

$ pip install --user pipenv

The --user option installs pipenv for the local user only. If you use it, you ensure that you won’t interfere with system-wide packages. Honestly, I just installed it globally without any problems.

Using pipenv

Now that you have pipenv installed, let’s try it!

First, create a directory. I called mine myenv. Now cd into the directory and install your first dependency with pipenv:

$ pipenv install requests

What happens next is:

  • pipenv detects there’s no virtual environment yet, so it creates one.
  • It installs requests.
  • It creates two files: Pipfile and Pipfile.lock.

The screenshot in @fig:pipenv_in_action shows pipenv in action on my system.

Pipenv in action on the author's system
Pipenv in action on the author's system

After it’s done, you can optionally enter the virtual environment with:

$ pipenv shell

This is optional because you can also launch a command in the virtual environment like this:

$ pipenv run <your command>

Where is My Virtualenv?

I don’t like black boxes. I want to know what’s going on. So let’s first inspect Pipfile. It should look like this:

[[source]]
 name = "pypi"
 url = "https://pypi.org/simple"
 verify_ssl = true

 [dev-packages]

 [packages]
 requests = "*"

 [requires]
 python_version = "3.6"

As you can see, Pipfile contains our top-level requirements without specifying a version number because we didn’t specify it. Now take a look at Pipfile.lock. It’s a big file, so I’ll just show a snippet here to illustrate:

...
"requests": {
  "hashes": [
    "sha256:43999036bfa82904....",
    "sha256:b3f43d496c6da..."
  ],
  "index": "pypi",
  "version": "==2.23.0"
},
...

Requests and all of its dependencies are listed, including the version numbers. Basically, this is an advanced version of the output of pip freeze. These are the versions we worked with. It’s the combination of packages that are guaranteed to work.

But where’s the virtual environment? If you look closely at the screenshot in @fig:pipenv_in_action, you see the virtualenv is created in ~/.local/share/virtualenvs/myenv-mAvwj65b/. We could have asked pipenv too with pipenv --venv.

Since we are already familiar with the structure of a virtual environment, you’ll notice that this one is no different. It’s just located somewhere else, outside of your project structure. This way, you don’t need to exclude it from your version control system too.

If you create another virtual environment with the same name in another location, it won’t be reused. The hash in the directory name is, I presume, based on the path to your virtualenv. Pipenv uses it to find the correct one despite double names.

Separating Development Packages

With pip, you are only able to separate development requirements from production requirements by using a second requirements file (e.g. requirements-dev.txt). With pipenv, you can simply use the --dev option:

$ pipenv install pytest --dev

Another developer that starts working on your project can install all requirements, including the developer requirements, using:

$ pipenv install --dev

Dependency Management

pipenv attempts to install all sub-dependencies required by your core dependencies. Conflicts might arise, though. If package A requires at least version 1.10 of package B, while package C requires version 1.9, you have a problem. pipenv can’t solve it for you, but it will warn you about it and refuse to continue by default. It also offers a nice insight into your dependencies with the pipenv graph command:

$ pipenv graph
pytest==5.3.5
  - attrs [required: >=17.4.0, installed: 19.3.0]
  - importlib-metadata [required: >=0.12, installed: 1.5.0]
    - zipp [required: >=0.5, installed: 3.1.0]
  - more-itertools [required: >=4.0.0, installed: 8.2.0]
  - packaging [required: Any, installed: 20.3]
    - pyparsing [required: >=2.0.2, installed: 2.4.6]
    - six [required: Any, installed: 1.14.0]
  - pluggy [required: >=0.12,<1.0, installed: 0.13.1]
    - importlib-metadata [required: >=0.12, installed: 1.5.0]
      - zipp [required: >=0.5, installed: 3.1.0]
  - py [required: >=1.5.0, installed: 1.8.1]
  - wcwidth [required: Any, installed: 0.1.8]
requests==2.23.0
  - certifi [required: >=2017.4.17, installed: 2019.11.28]
  - chardet [required: >=3.0.2,<4, installed: 3.0.4]
  - idna [required: >=2.5,<3, installed: 2.9]
  - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.8]

Detection of Security Vulnerabilities

pipenv can scan your packages for security vulnerabilities. It does so by using the safety package. Enter:

$ pipenv check

and pipenv will scan your dependency graph for packages with known security vulnerabilities!


If you liked this page, please share it with a fellow learner: