Took me some thinking to figure out where this design is not optimal.
The first step is to separate the problems. The content addressable object storage and storing the refs. libgit2 has two backends for those already.
Then use appropriate distributed dbs for the two backends. With large scale something like Cassandra for the objects (AP) and FoundationDB (CP) for the refs (or the equivalent Amazon services).
For performance the objects need to be cached locally and requests routed to nodes where the objects are cached locally.
The first step is to separate the problems. The content addressable object storage and storing the refs. libgit2 has two backends for those already.
Then use appropriate distributed dbs for the two backends. With large scale something like Cassandra for the objects (AP) and FoundationDB (CP) for the refs (or the equivalent Amazon services).
For performance the objects need to be cached locally and requests routed to nodes where the objects are cached locally.