Poetry: Dependency and virtual environment manager
Poetry is one of the most popular dependency and virtual management tool for python. A dependency manager allows you to specify, install, update, and manage external libraries or packages (dependencies) that a project relies on. For example, this is a simple Poetry requirements file that uses Python 3.11 and the requests and numpy Python packages.
[tool.poetry.dependencies]
python = "^3.11"
requests = "^2.25.1"
numpy = "^1.19.5"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
By using Poetry to pin your dependencies, you always ensure that you install the correct version of the dependencies that your projects work with. Poetry, by default, saves all its requirements in pyproject.toml file.
Another massive advantage of using Poetry is that it creates a new Python virtual environment in which it installs the specified Python version and requirements. A virtual environment allows you to isolate your project’s dependencies from your global Python dependencies and other projects. By doing so, you ensure there are no version clashes between prjects.
For example, let’s assume that Project A needsnumpy == 1.19.5 and Project B needsnumpy == 1.26.0 If you keep both projects in the global Python environment, that will not work, as Project B will override Project A’s numpy.
installation, which will corrupt Project A and stop it from working. Using Poetry, you can isolate each project in its own Python environment with its own Python dependencies, avoiding any dependency clashes.
Installation
You can install Poetry from here. Once Poetry is installed, navigate into your repository and run the following command to install all the necessary Python dependencies:
poetry install
This command knows to pick up all the dependencies from your repository that are listed in the
pyproject.toml and poetry.lock files. After the installation, you can activate your Poetry environment by running
poetry shell
in your terminal or by prefixing all your CLI commands as follows:
poetry run <your-shell-command>
Poetry locks down the exact versions of the dependency tree in the poetry.lock file based on the definitions added to the
project.toml file. While the pyproject.toml file may specify version ranges (e.g. requests = "^2.25.1"), the poetry.lock file records the exact version (e.g. requests = "2.25.1") that was installed. It also locks the versions of sub-dependencies (dependencies of your dependencies), which may not be explicitly listed in your pyproject.toml file. By locking all the dependencies and sub-dependencies to specific versions, the poetry.lock file ensures that all project installations use the same versions of each package. This consistency leads to predictable behavior, reducing the likelihood of encountering “works on my machine” issues.
Other tools similar to Poetry are Venv and Conda for creating virtual environments. Still, they lack the dependency management option. Thus, you must do it through Python’s default requirements.txt files, which are less powerful than Poetry’s lock files. Another option is Pipenv, which feature-wise is more like Poetry but slower, and uv, which is a replacement for Poetry built in Rust, making it blazing fast.
uv has lots of potential to replace Poetry, making it worthwhile to test out here.
Back to code!