zanie
Joined 4 years ago
Joined 4 years ago
But this was just one example optimization, we do other low-level things, like zero-copy deserialization from our cache. The point is not that we do specific things, but that we have more levers to pull to improve performance. It's great to see all the improvements happening in pip performance regardless :)
- https://docs.astral.sh/uv/reference/environment/#uv_concurre...
- https://docs.astral.sh/uv/reference/environment/#uv_concurre...
> The only advantage of uv is to have support for parallel async extraction.
This isn't true, the biggest speed ups are for the warm cases where we've already unpacked the files into the cache, as notatallshaw mentions above. We can also make low-level optimizations during resolution, e.g., in version parsing, that are not possible in pure Python code.
> If pip extracted multiple files/wheels in parallel without being blocked by the GIL, pip could easily match or outcompete uv.
I'm a bit confused by these claims about the GIL? The expensive IO operations release the GIL.
> When the user edits one file, all files are modified simultaneously
This is why we default to reflinks or copy-on-write semantics when creating environments, not all file systems support it but it's becoming more common.
> Hypothetically, a simple deduplication of binary files (.dll .so) should achieve 50% of the savings without significant drawbacks
We also explored this (see https://github.com/astral-sh/uv/pull/19694) and the linked pull request has a table comparing to this strategy.
> We have optimization for empty files (0 bytes) because there is nothing to write and checksum.
Interesting, I would be very surprised if this made a significant difference? but I'll take a look.
If you're running Python scripts, I'd recommend using `uv lock --script <path>` to generate a lockfile — we don't do that by default for scripts yet but that will avoid unexpected upgrades.
(I work on uv)
It's also non-trivial to implement because of how our cache design differs from pip's.
(but as a note, the initial `uv lock` will already attempt to choose the highest possible versions and `uv lock --upgrade` will do so again, requests for `uv upgrade` are generally centered around changing the constraints in the `pyproject.toml`)
There have been several threads about us collaborating or converging, but there are no concrete plans to do so at this time. We share some Rust crates, e.g., Pixi uses our internals to solve the PyPI-side of their dependencies, and we occasionally fix or change things for each other.
TIL. I ran some benchmarks and confirmed this is the case for many small files as you'd see in wheels — the GIL is released in a meaningful way for larger files though. Thanks!
> This might not give the same result for uv though.
Yeah, I built a prototype and ran some benchmarks. It makes a big difference if the entire wheel is empty files but for any real world examples it's within noise of the baseline.