ByronBates
Joined 8 years ago
Joined 8 years ago
I couldn't resist and threw `gitoxide` at it, and it turned out to be more than 2x as fast (even though it uses way more CPU to do that, there is definitely room for improvement).
The PR which adds the `db-gen` program: https://github.com/jmforsythe/Git-Heat-Map/pull/6
That said, here is how to receive a pack from the remote and resolve it: `mkdir out && gix -v no-repo pack receive https://github.com/Byron/gitoxide out`.
From my experience and measurements as the author of `gitoxide` both the parsing of commits as well as the binary search to find the object are negligible costs here. The majority of the time is spent in `zlib` which has to inflate objects and deltas prior to use. This makes me wonder what `zstd` would do to these kinds of workloads.
In any case, the object database decoding performance is the bottleneck when traversing commits, and one can expect to get about 120000 commits per second on the linux kernel pack on a modern CPU core like M1 with `gitoxide`s implementation.
Besides that, I find the explanation of the commit-graph file interesting and how it relates to commit-graph queries. There is so much to learn, and so much still to implement :).
The same applies when checking for changes on disk - right now git does not do that in parallel. These are low hanging fruit that I plan to pick for my own sake.
There is another contributor who is very interested in increasing pack performance, which will directly impact fetch and pull speeds. Judging from my experience with packs thus far, I believe a lot of options are still to be tapped in that field as well. Rust will open it up for contributions and experiments to a greater audience, so I would hope that this will go way beyond of what I can do.
Even though this project is not very contributor friendly right now, I will be working on improving this so more people can join in (see https://github.com/Byron/gitoxide/issues/8).
Please note that everything I name can certainly be fixed by some tool that already exists, and that's great. It's just that I would like to have 'this one tool' that gets most things right and be happy with it alone.
That said, here is my list:
* adding individual files to the staging area/index
* looking at the commit history and copying individual commit hashes
* pulling while assuring no merge commit is created, or: simplified trunk-based development
* resumable clones
* seeing what's going on during all remote operations - here in China GitHub is incredibly slow most of the time and connections can be flaky
It is my hope that the quality and stability will be high enough to convince people [in the Rust ecosystem] to move away from libgit2 and instead contribute to the maintenance and development of gitoxide, rewarding everyone with better performance and improved usability.