Have you ever been to dependency hell? Well, I have for sure. I have many smaller projects that use Python, but they were all created at different times throughout my career. One new project uses the latest Python version, one slightly older project uses Python 3.5, and an ancient project uses Python 2.8. I know, and I’m a bit ashamed that I haven’t updated it yet.
You might wonder, “Well, that is only the Python version; it can’t be too hard to fix.” And you are right; however, I used packages such as JSON, requests, NumPy, and so on in the projects. Having them globally means that you need to install and reinstall different versions depending on what you are working on, and it will become a “dependency hell” after a while and break everything.
That is why, in this blog post, I’ll talk about setting up Poetry to handle dependencies and isolate projects from each other—a real savior of the hell of dependency management!
If you want to read and instead watch a video, don’t worry, I got you covered! Check out the following youtube video.
Install Poetry on your machine
Windows
- Open PowerShell as an administrator
- Install Poetry
(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -
- Restart the shell
- Verify installation
poetry --version
Linux
- Open Terminal
- Install Poetry
curl -sSL https://install.python-poetry.org | python3 -
- Reload your shell
source ~/.bashrc
- Verify installation
poetry --version
macOS
- Open Terminal
- Install Poetry
curl -sSL https://install.python-poetry.org | python3 -
- Update either .bash_profile or .zshrc with the following content:
export PATH="$HOME/.local/bin:$PATH"
- Reload your shell
source ~/.bashrc
- Verify installation
poetry --version
Lets start using Poetry!
A nice feature of Poetry is that we can use it for existing projects and new projects that we want to start. Here’s a guide to handling each scenario:
Existing project
-
Navigate to the project root directory
-
Initialize Poetry
poetry init
This will launch an interactive setup of an ./pyproject.toml. Input the values that you want. This is all going to be saved in an ./pyproject.toml file later. However, when you get prompted to install main dependencies and dev dependencies, I would recommend answering ‘no’ since you’ll need to learn how to handle them after setup is completed. Now there is a pyproject.toml file in the root directory. It should look something like the following, with some data that was input interactivly.
[tool.poetry]
name = "my-project"
version = "0.1.0"
description = "My project description"
authors = ["Arjan <[email protected]>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.12"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
If you already have a requirements.txt file, you can use Poetry to read it and add the dependencies to pyproject.toml by running:
poetry add $(cat requirements.txt)
Otherwise, we need to add the dependencies manually. For example:
poetry add requests openai
Now that we can look at the ./pyproject.toml file, the dependencies are added to the project. However, they’re not yet installed.
...
[tool.poetry.dependencies]
python = "^3.12"
requests = "^2.31.0"
openai = "^1.6.0"
...
Let’s install the Poetry dependencies.
poetry install
After the installation is complete, notice how a poetry.lock file appears in the root directory. The ./poetry.lock is crucial and an important part of Poetry. Poetry calculates the specific versions of each dependency (and their dependencies). These exact versions are then written into the ./poetry.lock file. This ensures that every time someone else installs the project’s dependencies using poetry install, the same versions of the dependencies are installed. This consistency is crucial for avoiding “it works on my machine” problems. Never update this manually! Always let Poetry handle the ./poetry.lock file.
New project
- Navigate to the directory where you store all your projects
- Create a new project using Poetry
poetry new my-new-project
Notice how this is not interactive!
- Navigate to the project
cd my-new-project
As we can see, there are more files and directories. This is because Poetry doesn’t know what files are existing or not in the project, so it can’t generate everything.
However, if it’s a new project, then it knows the project should at least contain a ./README.md, ./pyproject.toml and ./tests directory
- Add the dependencies
poetry add requests openai
- Install the dependencies
poetry install
And we’re done! We now have Poetry setup for both an existing project and a new project! This is just the surface of what Poetry can do. There’s so much more that happens under the hood and more functionality that we can explore! For example, Poetry automatically manages virtual environments for projects, updates dependencies, and even assists with building publishing packages for repositories like PyPI.
Final thoughts
Poetry is a powerful dependency manager, which is crucial for any project. Are there other dependency managers? Of course! However, the simplicity and out-of-the-box functionality of Poetry are hard to beat. If you want to build a package for Python that anyone can use, Poetry is for you! If you want to create a CLI command with a click? Poetry is for you!
I hope after you read this post that you will incorporate Poetry into existing projects and new projects to avoid dependency hell.
