amirouche
Joined 14 years ago
Joined 14 years ago
It needs polish here and there, but the results hold up, and some of its quirks surprised me in good ways. No ads, no tracking, no personalisation.
The corpus:
English (93M docs, 2.1 TB): Stack Overflow, arXiv, Wikipedia, Hacker News and Lobsters up to 2020
French (18M docs, 0.5 TB): theses.fr, Persée, Gallica, Wikisource, Wikipédia
How it’s built, since that’s probably the interesting part here:
Everything is Chez Scheme. The storage layer is an ordered key-value store (it implements SRFI-167/168, which I wrote) running as an LSM tree.
The C parts: libtls, liburing, picohttpparser, libc. French and English are separate indexes. Mixing them corrupted document frequencies in an earlier version, because French and English words collide after stemming.
Known weaknesses: ranking is the next big piece of work.
What would help most: a query where it failed you, or a source you think belongs in the corpus and why. I’m especially curious which corpora people would put in a “technical history” index.
The code isn’t public yet. I want to see what people actually use before I freeze the design.
There is json output: https://h9p3r.fr/?q=biological+intelligence+future+&lang=en&...
Two small PRs, each based on a single prompt, resulted in a CLI that captures the core idea. The goal was to make it relatable to others—let me know if it resonates with you.
Observations & Process: Claude asked a lot of clarifying questions, while ChatGPT "hallucinated" a broader "cognitive ecology" (likely because I’d mentioned the project under the name "mobius"). The final README reflects a blend of ChatGPT’s grandiose vision and Claude’s more grounded approach. It frames ouverture as a symbiotic post-LLM relationship—a bridge between existing knowledge (including code) and post-LLM AI systems.
The Bigger Picture: If ouverture succeeds, it could become an infrastructure like npmjs—but with less friction, less drama, and fewer barriers. The irony? The README’s vision remains relevant even without LLMs. The core idea—content-addressable, multilingual code—stands on its own.
Origins & Goals: This idea has been brewing for over a decade. My original goals were:
Code as a reusable resource: Write a function, store it, forget it, and retrieve it later—dependencies and all—without the hassle of reinventing wheels (e.g., leftpad or buried helper functions). Lowering cognitive barriers: Enable people to contribute to code without requiring English proficiency, aligning with the "think globally, act locally" ethos.
Inspirations: Key projects that shaped this thinking:
- Unison (content-addressable code) - Abstract Wikipedia (multilingual knowledge) - Situated Software
ouverture.py is my answer to [this Lobste.rs discussion on easing the production of micro-libraries](https://lobste.rs/s/ebbaed/how_ease_production_micro_librari...).
Note: An attacker needs to be stronger than one defense, that is unlike being stronger than the sum of all defenses.
Note2: What we now see as a collective ie. a generated reality, existed before now, and is documented, the following comes to mind 1984, Good Bye Lenin, The Village M. Night Shyamalan, and IIRC Gate to Avalon, by Mamoru Oshii.
edit: added squared brakets.