notatallshaw
Joined 3 years ago
Joined 3 years ago
This one being downloading to an offline wheelhouse and installing from that.
The other one being having a shared named global environment ;o).
P.S. I'll have to remove this as an important feature nab has that uv doesn't when I make the announcement nab is no longer experimental, aha.
This makes a lot of sense because pubgrub makes heavy use of version comparison, compared to simple DFS algorithms like the one resolvelib uses. A lot of Pubgrub optimizations come from finding clever ways to not need to keep comparing versions.
> But this was just one example optimization, we do other low-level things, like zero-copy deserialization from our cache. The point is not that we do specific things, but that we have more levers to pull to improve performance.
Oh yes, I agree with the general point, I was just picking on the specific example for a fun exploration of performance optimizations.
Also, FWIW, I have in my professional career, not OSS work, implemented zero-copy deserialization from cache in pure Python, there are many surprising levers in Python when you are willing to explore the weird corners of the standard library.
As a complete aside, uv can and does do this, but for this particular optimization I'm not sure how much absolute time it ends up saving compared to pure Python in real world resolution scenarios.
uv's total memory usage isn't that much leaner than pip's, and for parsing speed it turned out that the library pip uses, packaging, was just very unoptimized at the time uv launched. This has been significantly addressed since then:
* We did a lot of work to make version parsing twice as fast: https://iscinumpy.dev/post/packaging-faster/
* Since that blog post I made typical version parse three times faster on top of that: https://github.com/pypa/packaging/pull/1082
* Also since that blog post version filtering has gone through multiple optimizations and in some cases will be more than 30x faster e.g. https://github.com/pypa/packaging/pull/1105, https://github.com/pypa/packaging/pull/1111, https://github.com/pypa/packaging/pull/1120
At this point large dependency resolves in pip are spending very little of their time doing things in packaging, like version parsing. The main non-IO time spent in large resolves is now in the core resolver, resolvelib, which I hope to one day replace with my experimental resolver nab: https://github.com/notatallshaw/nab. Nab scales to large resolves much more efficiently than resolvelib (in fact I've cross-ported some of the algorithmic efficiency gains to uv already ;o)).
But it has always had two major issues:
1. No way to reproduce exact distributions for a "download" command (there is no uv equivalent of "pip download")
2. For people with a lot of different environments the cache grows significantly more than pip
I'm interested to see, at least anecdotally, if this significantly improves 2, then we can perhaps have a two layer caching strategy without the significant disk space cost.
If Astral/OpenAI are willing to support uv pip in the long term, why do you think they should use a stick as well as a carrot to get people on uv's top level interface?
While uv's standard workflows probably works for 80+% of people, at least if they learn them, I certain do a lot of stuff with uv pip and pip that can't be replicated.
Also both uv pip and pip support installing from pylock.toml files, which are fully reproducible.
Hi, I'm a pip maintainer, can you please report any issue you think is happening in recent versions of pip: https://github.com/pypa/pip/
I particularly work on performance and stability and think we've been making big strides over the last two years
> getting everyone to install private venv's for every app, leading to missing security updates, because updates just break everything.
venv's stop your OS tooling from breaking, Linux distro maintainers pushed Python packaging to not allow installing into the system packages by default, and they weren't wrong.
All these other problems are solved with proper use of a lock file (which are now standardized, and you can generate and install from them in the latest version of pip), and/or use a project tool (uv, poetry, etc.).
FYI, pip added cooldowns in 26.1:
* https://discuss.python.org/t/announcement-pip-26-1-release/107108
* https://ichard26.github.io/blog/2026/04/whats-new-in-pip-26.1/
To use: * CLI: pip install --uploaded-prior-to=P1D ...
* Env Var: PIP_UPLOADED_PRIOR_TO=P1D pip install ...
* Config: pip config set global.uploaded-prior-to P1DIn pip 26.1 (release scheduled for April 2026), it will support the day ISO-8601 duration format, which uv also supports, so you will be able to do --uploaded-prior-to=P3D, or via env vars or config files, as all pip options can be set in either.
> If you've identified a security issue with a project hosted on PyPI Login to your PyPI account, then visit the project's page on PyPI. At the bottom of the sidebar, click Report project as malware.
What uv does is parallelize the final download of packages after resolution, and batch pre-fetch metadata during resolution. I don't think these benefit from async, due to their batch nature classic multi-threaded download pools are probably the better solution, but I could be wrong!
Experiments have been done on the former in pip and didn't find much/any improvement in CPython, this may change in free threaded CPython. For the latter we currently don't have the information from the resolver to extract a range of possible metadata versions we could pre-range, I am working on this but it requires new APIs in packaging (the Python library) and changes to the resolver, and again we will need to benchmark to see if adding pre-fetching actually improves things.
The original request to add a concurrency variable in uv was requested by me because, among other things, I wanted to measure the difference between uv and pip: https://github.com/astral-sh/uv/issues/3311
I'm not really following this performance discussion as I think it's gone off the rails.