Bootstrapping Development with Poetry

September 24, 2021

Overview

Poetry is a library for dependency management and packaging in Python (Documentation).

Key Features

  • a single configuration file (pyproject.toml)
  • efficient dependency resolution (to resolve compatible versions of immediate and transitive dependencies)
  • deterministic builds by locking the versions of both immediate and transitive dependencies in a poetry.lock file
  • seperate dependencies within groups and based on context (development dependencies, production dependecies, operating system constraints)

Installation

Install Poetry

Poetry provides a custom installer that isolates itself from the rest of your system (Installation).

curl -sSL [https://install.python-poetry.org](https://install.python-poetry.org) | python3 -

The installer installs the poetry CLI tool to Poetry's bin directory. This location depends on your system:

  • $HOME/.local/bin for MacOS/Linux
  • %APPDATA%\Python\Scripts on Windows

If the above directory is not on your PATH, you will need to add it manually if you want to invoke Poetry from the command line using poetry.

Update Poetry

Update poetry to the latest stable version:

poetry self update

Uninstall Poetry

Remove poetry from your system:

curl -sSL [https://install.python-poetry.org](https://install.python-poetry.org) | python3 - --uninstall

Package Configuration

All package configuration is defined in pyproject.toml, a new configuration file format introduced in PEP 518 to specify the minimum build system requirements for Python projects. The pyproject.toml file replaces setup.py, requirements.txt, setup.cfg and MANIFEST.in, and provides a standard and central location for build and non-build tools to be defined (resulting in less configuration files floating around).

Poetry Configuration

Poetry can be configured to customise the way it handles caching and virtual environment creation. Here is a list of the available Poetry configuration options. These configuration settings can be defined globally or locally.

View Current Configuration Settings

To list the current configuration settings:

poetry config --list

Update Configuration Settings

To add or update a global configuration value, pass the value after the setting's name:

poetry config <setting> <value>

To add or update a local configuration value, pass the value after the setting's name using the --local flag:

poetry config <setting> <value> --local

By default, Poetry installs virtual environments for each project in a global cache directory. It may be useful to install virtual environments in the root directory of projects, which can be achieved as follows:

poetry config virtualenvs.in-project true

Usage

Poetry uses both the pyproject.toml and poetry.lock files to synchronise the dependencies and versions of your project. The pyproject.toml file specifies the minimum build system requirements for your project, and the poetry.lock file contains the current hashed versions of all immediate and transitive dependencies.

A basic pyproject.toml file looks like this:

[tool.poetry]
name = "my-package"
version = "0.1.0"
description = "My new package using Poetry"
authors = ["Luke Miloszewski <lukemiloszewski@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.6"

[tool.poetry.dev-dependencies]
pytest = "^3.4"

Initialise a Project

If a pyproject.toml file doesn't exist in your project, you can create one interactively:

poetry init

If a pyproject.toml file does exist in your project, you can install the pre-defined dependencies in a virtual environment:

poetry install

The location of the virtual environment is based on the Poetry configuration set for virtualenvs.path. Furthermore, as dependencies are installed, they are added to the poetry.lock file. It is good practice to commit the poetry.lock file to your version control system. This ensures that anyone can install your package in a deterministic manner.

Update Dependencies

To add a dependency:

poetry add pandas

Use the --dev flag if the dependency is for development only. Then, when you want to initialise a project without any development depenencies:

poetry install --no-dev

To add a GitHub dependency:

poetry add git+[https://github.com/project/repo.git](https://github.com/project/repo.git)

Using a Custom Repository

To publish a package to a custom repository, add the repository to the Poetry configuration:

poetry config repositories.foo [https://private-repository/simple/](https://private-repository/simple/)
poetry config http-basic.foo username password

To install a package from a custom repository, add the repsitory to pyproject.toml:

[[tool.poetry.source]]
name = "private-repository"
url = "[https://private-repository/simple/"](https://private-repository/simple/")

Publish your Package

To publish a package to a remote repository:

poetry publish --build

The --build flag will build your package before publishing it. Furthermore, the default repository will be pypi. If you want to publish to a custom repository, you can use the --repository flag to specifiy the repository name.