benmathes

Don't attribute to malice what is better explained by coordination headwinds in extremely large companies.

The engineering around tooling wasn't remotely the issue. It's getting all the (thousands?) data researchers mostly iterating on fine tuning datasets that would get bristly if they couldn't work outside version control in a python notebook iteratively tweaking their dataset that processed and reprocessed a few datasets until a threshold was reached.

The only _guaranteed_ chains of custody are down at the compute job and file read level. Which in a massively distributed computing job is... [redacted] nodes reading [redacted] fanouts of "datasets" that is just an abstraction over [redacted] individual files.

There's no malice here. Just way way way more complex than you'd first think.

All of your points are valid, and believe me I was trying to make them. The problem is one of culture. Most of the people doing this kind of work didn't like version control, and their work was really just running notebooks (like iPython or Google Colab) until a number was good enough and they'd submit the file for inclusion into training runs.

You can call it a policy failure, but these people were in very high talent demand and so top down dictates would risk "X people leaving lab Y for lab Z" headlines and morale hits.

I am not saying this is good. I am telling you that on the ground it is so much messier than it should be.

I worked in the tracing and tracking all the thousands of data sets that got tweaked and permuted and changed hands between thousands of researchers and data engineers at a major lab. The data that goes into training runs is permuted so much from the OG data that tracing the lineage is not trivial (dramatic understatement).

And the difficulty is harder than just the extreme scale of text searching. but also explodes with organizational difficulty since there are so many people tweaking/shifting data independently upstream of the actual training run, and no they will not all add the telemetry you wish they did.

In the ideal, should it be this hard? Well, no, but that's org wrangling for you.

that many people likely live in different timezones, or have conflicting meetings, etc. etc.

One model I've seen is moving the review stage to the designs, not the code itself.

I.e. have a `planning/designs/unbuilt/...` folder that contains markdown descriptions of features/changes that would have gotten a PR. Now do the review at the design level.

in 2020 we lived through the extreme left's remedial lesson in public safety. Now we're living through the MAGA right's remedial lesson in trade policy.

(Most MAGA right know this, don't care, see tariffs as a hammer. But they are hitting all of us)

I see the value here. The problem isn't just the search; it's the trust. The biggest hurdle for Zenode won't be the tech, but convincing an engineer that your AI's summary of a footnote is accurate enough to risk a $10,000 board spin. That's a high bar.

I'd argue the core value isn't just a better search or a faster reader. It's about providing a verified, reliable source of truth. This brings up a key tension: you say the AI isn't yet at your co-founder's level of accuracy, but is that precisely the level of confidence required to replace an engineer's manual check? How do you close that gap? You've got the data, but the trust factor is a different threshold?

I.e. maybe you've built the tool to make the problem faster, but the real win would be a tool that makes the problem safer? The killer feature might not be more speed, but rather a confidence score on every AI-generated fact, with a clear path to the source document so an engineer can verify it. It’s not about avoiding the document entirely; it’s about having a better starting point and knowing exactly what to double-check.

"personal" doing a lot of work there :-)

(And I'd be envious of your impact, of course)

Searched for "innovator's dilemma" in the article, didn't find it.

The crux is not whether the extremely rich incumbent (google) will have better infrastructure, but whether that is the field of competition that matters.

With golf, there are other dimensions course designers could use to make the pro game less about distance. Tighter fairways, faster greens, etc.

I suspect this may result in more pitches away to these batters so they have to make contact on the narrower end of the bat.

I.e. it's a move that may well have a counter move.