@charliermarsh: This talk that I gave at Jane Street a year ago is by far the one that is referenced to me most in-person. People seem …

This talk that I gave at Jane Street a year ago is by far the one that is referenced to me most in-person. People seem to like it!

https://t.co/GPFnL34m08

TL;DR: Charlie Marsh explains how uv, a Rust-based Python package manager, solves dependency resolution with a CDCL SAT solver and a unified, portable lockfile, delivering extreme speed by rethinking the entire toolchain from scratch.

Introduction: Why Build Yet Another Python Package Manager?

Charlie Marsh, founder of Astral (known for the Ruff linter and formatter), presented at Jane Street about uv – a fast, all-in-one Python package and project manager. Released in February, uv already sees 16 million monthly downloads and accounts for over 10% of requests to PyPI. The talk covers:

• What uv is and why it was built
• Technical challenges in creating a state-of-the-art resolver
• Performance case studies that change how users work with Python

The core motivation: Python’s packaging ecosystem is fragmented (pip, pipx, pyenv, virtualenv, poetry, etc.). uv aims to be the single static binary that does everything – install Python, create virtual environments, resolve dependencies, install packages, and build distributions – mimicking Rust’s unified Cargo experience.

The Power of a Fully Unified Stack

Unlike previous attempts at Python package managers, uv is not built on top of any existing tool. It implements everything from scratch:

• No dependency on pip or setuptools
• No inherited limitations from legacy Python packaging
• A single lockfile format (like uv.lock) that captures every detail: package name, version, source, dependencies, SHA, and file size

This unification allows uv to treat virtual environments as cheap, disposable resources. “Destroying and recreating a virtual environment is extremely fast – we try to make it so cheap that you don’t care about keeping it pristine.”

How uv Installs Packages: The Lifecycle

When you run uv pip install or uv sync, two main interfaces are available:

1. pip-compatible CLI – uv pip install for drop-in replacement (e.g., replacing pip install pydantic)
2. Higher-level commands – uv sync / uv lock for project-based workflows where you declare dependencies and let uv manage everything

The lifecycle:

1. Find a Python interpreter – uv doesn’t ship Python, so it locates an existing one (sans interesting details).
2. Discover requirements – read from command line or requirements.txt.
3. Resolve dependencies – parse requirements into a set of compatible versions. This is the heart of the package manager.
4. Generate a lockfile – a full graph with exact versions, sources, and checksums.
5. Create an installation plan – compare current environment state with the desired lockfile state, then update (uninstall old versions, install new ones).

Hard Problem #1: No Multi-version Support in Python

In Python, you cannot install two versions of the same package simultaneously (unlike Rust or Node). The import system uses a global cache keyed by module name, so you cannot have pydantic==1.* and pydantic==2.* at the same time.

Example: Suppose your project depends on vcr (which requires pydantic>=2) and an old langchain (which requires pydantic<2). This graph is unsolvable in Python. uv gives a clear error message: “Because your project depends on langchain which requires pydantic<2, and vcr requires pydantic>=2, these cannot be satisfied.”

If the user says “I accept any version of vcr”, uv must backtrack – test every version of vcr to find one that satisfies all constraints. This is essentially a boolean satisfiability (SAT) problem, which is NP-hard.

uv’s approach: It uses a CDCL (Conflict-Driven Clause Learning) SAT solver. This is a smart technique for exploring the dependency space, learning from conflicts, and pruning the search tree. “It’s exponential in the worst case, but in practice it’s fast enough for Python’s package universe.”

Hard Problem #2: Platform- and Version-Dependent Conditions

Python’s dependency specifiers are incredibly rich: importlib-metadata; python_version < "3.10", colorama; sys_platform == "win32", etc. This creates a huge number of conditional requirements.

Simpler approach: Solve for a specific target (e.g., Windows + Python 3.12) – just filter irrelevant constraints. But uv wants to generate a universal lockfile that works on any machine. That means the resolver must solve an enormous problem: for each package version, consider all possible combinations of platforms, Python versions, and feature flags, and ensure that the lockfile is valid everywhere.

“This is what makes the resolver the hardest part of uv.”

Performance That Changes Workflows

Speed is not just a benchmark score; it transforms how people use Python:

• Ruff precedent: Tasks that used to run only in CI now run as pre-commit hooks because they’re so fast.
• uv precedent: Creating a virtual environment and installing all dependencies from scratch takes seconds instead of minutes. Developers can now treat environments as ephemeral – destroy and recreate without fear.
• 10% of all PyPI traffic now comes from uv, despite only ~6 months since release.

The talk ends with case studies (not fully transcribed) demonstrating specific optimizations: faster download with parallel HTTP, streaming & caching of metadata, lockfile-based incremental installation, and Rust’s memory safety without a garbage collector.

Conclusion: A New Era for Python Tooling

uv represents a deliberate attempt to change the Python packaging landscape by offering a unified, blazingly fast drop-in replacement for the entire stack. Its adoption shows that the community craves simplicity and speed. As Charlie says, “When things are ten times faster, it changes your relationship with the tool.”

Source: @charliermarsh – “This talk that I gave at Jane Street a year ago…”